Hgc^erche  nr  eTTsr.1 

30Lr  lu  C£k!['!iL>  CriTIJLii 


DEFENCE 


DEFENSE 


Mapping  the  Relevant  of  Complex  Decision 
Making  to  Canadian  Forces  Land 
Operations 


Lisa  A.  Rehak,  Tamsen  E.  Taylor,  Lora  Bruyn  Martin 

Humansystems®  Incorporated 
111  Farquhar  St., 

Guelph,  ON  N1H  3N4 

Project  Manager: 

Lisa  A.  Rehak 

PWGSC  Contract  No.:  W771 1-0981 58/001 /TOR 
Call-up  8158-02 

Contract  Scientific  Authority: 

Jerzy  Jarmasz 
416-635-2000 


The  scientific  or  technical  validity  of  this  Contract  Report  is  entirely  the  responsibility  of  the 
Contractor  and  the  contents  do  not  necessarily  have  the  approval  or  endorsement  of  Defence  R&D 
Canada. 


Defence  R&D  Canada 

Contract  Report 


DRDC  Toronto  CR  2011-079 
March  2011 


Canada 


HUMAN  SYSTEMS 


DRDC  Toronto  CR-201 1-079 


MAPPING  THE  RELEVANCE  OF  COMPLEX  DECISION  MAKING 
TO  CANADIAN  FORCES  LAND  OPERATIONS 

by: 

Lisa  A.  Rehak,  Tamsen  E.  Taylor,  Lora  Bruyn  Martin 

Human  systems®  Incorporated 
1 1 1  Farquhar  St., 

Guelph,  ON  N1H  3N4 

Project  Manager: 

Lisa  A.  Rehak 


PWGSC  Contract  No.:  W771 1-0981 58/001 /TOR 
Call-up  8158-02 

On  Behalf  of 

DEPARTMENT  OF  NATIONAL  DEFENCE 


as  represented  by 

Defence  Research  and  Development  Canada  Toronto 
1133  Sheppard  Avenue  West, 

Toronto,  ON,  M3K  2C9 

Contract  Scientific  Authority: 

Jerzy  Jarmasz 
416-635-2000 


March  201 1 


HUMANSfSrfMS 


Author 

Original  signed  by  Lisa  Rehak 

Lisa  Rehak 

Humansyste/ns®  Incorporated 


Approved  by 

Original  signed  by  Jerzy  Jarmasz 

Jerzy  Jarmasz 
Contract  Scientific  Authority 


Approved  for  release  by 

Original  signed  by  Dr.  Stergios  Stergiopoulos 

Dr.  Stergios  Stergiopoulos 

Acting  Chair,  Knowledge  and  Information  Management  Committee 

Acting  Chief  Scientist 


The  scientific  or  technical  validity  of  this  Contractor  Report  is  entirely  the  responsibility  of  the 
contractor  and  the  contents  do  not  necessarily  have  the  approval  or  endorsement  of  Defence  R&D 

Canada. 


©  Her  Majesty  the  Queen  in  Right  of  Canada,  as  represented  by  the  Minister  of  National  Defence,  2011 
©  Sa  Majeste  la  Reine  (en  droit  du  Canada),  telle  que  representee  par  le  ministre  de  la  Defense  nationale, 


2011 


Abstract 


HUM  AN. S  YSTE MS 

ImvrporjItJ 


Challenging  decision  making  environments  such  as  those  experienced  by  the  Canadian  Forces  are 
commonly  being  characterized  as  “complex”  by  researchers  (e.g.,  Grisogono,  2010).  The  main  goal 
of  this  project  was  to  determine  whether  research  investigating  complex  decision  making  is 
relevant  to  the  decision  making  actually  experienced  by  Canadian  Forces  personnel,  and  how  that 
research  might  be  used  to  improve  Canadian  Forces  education  and  training  related  to  decision 
making.  Complex  decision  making  environments  are  characterized  by  requiring  a  series  of 
interdependent  decisions  in  a  context  that  changes  both  autonomously  and  as  a  function  of  the 
actions  of  the  decision  maker,  and  where  timing  is  a  key  element  (e.g.,  decision  makers  may  have 
to  act  at  particular  time  in  order  to  have  their  intended  effect).  Although  factors  identified  in  the 
complexity  literature  did  appear  to  play  a  strong  role  in  Canadian  Forces  decision  making,  further 
research  is  required  to  determine  the  relative  role  that  these  factors  play  in  increasing  decision 
making  difficulty.  Research  identified  additional  challenges  faced  by  Canadian  Forces  personnel 
that  were  not  noted  in  the  complexity  literature,  including  challenges  related  to  collaboration  and 
communication.  Other  areas  which  pose  significant  challenges  to  CF  personnel,  and  that  appear  to 
require  additional  education  and  training  include  planning  and  dealing  with  resource  challenges. 
Canadian  Forces  personnel  who  are  engaged  in  domestic  and  expeditionary  operations  appear  to 
encounter  the  highest  level  of  complexity  in  their  decision  making,  and  initial  education  and 
training  efforts  should  probably  focus  on  these  individuals  rather  than  individuals  engaged  in 
domestic  day-to-day  functions. 
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Les  chercheurs  (dont  Grisogono,  2010)  qualifient  generalement  de  «  complexes  »  les  milieux 
decisionnels  difficiles  comme  ceux  dans  lesquels  les  Forces  canadiennes  sont  appelees  a  servir.  Ce 
projet  avait  pour  but  premier  de  determiner  si  l’etude  de  processus  decisionnels  complexes  serait 
utile  a  la  prise  de  decisions  qui  constitue  la  realite  du  personnel  des  Forces  canadiennes,  et 
comment  ces  travaux  pourraient  servir  a  ameliorer  l’education  et  l’instruction  des  militaires 
canadiens  en  ce  qui  concerne  la  prise  de  decisions.  Les  milieux  decisionnels  difficiles  exigent  une 
serie  de  decisions  interdependantes,  dans  un  contexte  qui  change  a  la  fois  de  fag  on  autonome  et  en 
fonction  des  mesures  que  prend  le  decideur  et  dont  la  synchronisation  est  primordiale  (p.  ex.,  les 
decideurs  peuvent  devoir  agir  a  un  moment  en  particulier  afin  d’obtenir  l’effet  souhaite).  Les 
facteurs  releves  dans  la  documentation  sur  la  complexite  semblaient  effectivement  exercer  un  role 
important  dans  le  processus  decisionnel  des  Forces  canadiennes,  mais  d’autres  etudes  s’imposent 
afin  de  determiner  le  role  relatif  qu’exercent  ces  facteurs  par  rapport  a  la  difficulte  de  la  prise  de 
decisions.  Les  etudes  ont  releve  d’autres  defis  auxquels  se  heurte  le  personnel  des  Forces 
canadiennes  qui  n’etaient  pas  mentionnes  dans  la  documentation  sur  les  decisions  complexes,  y 
compris  des  defis  lies  a  la  collaboration  et  a  la  communication.  D’autres  secteurs  qui  posent  des 
defis  importants  aux  membres  des  FC  et  dans  lesquels  une  plus  ample  formation  semble  necessaire 
sont  notamment  la  planification  et  les  difficultes  liees  aux  ressources.  Comme  les  membres  des 
Forces  canadiennes  affectes  a  des  operations  nationales  et  expeditionnaires  semblent  etre  appeles  a 
prendre  les  decisions  les  plus  complexes,  les  premiers  efforts  d’education  et  d’instruction  devrait 
probablement  viser  ce  groupe  de  personnes  plutot  que  les  militaires  qui  exercent  des  fonctions 
courantes  au  Canada. 


Page  ii 


Human.s'v.stew.s1  Incorporated 


Executive  Summary 


HUM  AN. S  YSTE MS 

ImvrporjItJ 


Mapping  the  Relevance  of  Complex  Decision  Making  to  Canadian  Forces  Land 
Operations. 

Lisa  A.  Rehak,  Tamsen  E.  Taylor  and  Lora  Bruyn  Martin,  Humansystows®  Incorporated; 
DRDC  Toronto  CR2011-079;  Defence  R&D  Canada  -  Toronto;  March  2011. 

Challenging  decision  making  environments  such  as  those  experienced  by  the  Canadian  Forces  (CF) 
are  commonly  being  characterized  as  “complex”  by  researchers.  Researchers  have  proposed  that 
contemporary  military  operations  exhibit  characteristics  that  have  been  studied  extensively  in 
domains  such  as  chemistry,  physics,  and  biology,  and  research  findings  in  these  domains  could  be 
applied  to  the  study  of  CF  decision  making.  The  main  goal  of  this  project  was  to  determine  whether 
research  investigating  complex  decision  making  is  relevant  to  the  decision  making  actually 
experienced  by  Canadian  Forces  personnel,  and  how  that  research  might  be  used  to  improve 
Canadian  Forces  education  and  training  related  to  decision  making. 

Complex  decision  making  environments  are  characterized  by  requiring  a  series  of  interdependent 
decisions,  in  a  context  that  changes  both  autonomously  and  as  a  function  of  the  actions  of  the 
decision  maker,  and  where  timing  is  a  key  element.  In  this  project,  we  performed  a  brief  review  of 
the  complexity  literature,  and  identified  components  which  influence  the  difficulty  of  decisions  in 
complex  environments.  The  project  focused  on  five  of  these  factors  related  to  complexity, 
including: 

1)  Connectivity:  Things  in  the  environment  influence  one  another  in  complicated  and 
unpredictable  ways, 

2)  Dynamics:  The  system  has  aspects  that  unfold  over  time.  For  example,  the 
environment  changes  over  time  even  when  you  do  nothing;  the  rate  at  which  things 
change  may  be  variable;  there  may  be  delays  between  actions  and  effects, 

3)  Multiple  conflicting  goals:  Flaving  to  achieve  multiple  objectives  which  may  not  be  all 
achievable  at  the  same  time, 

4)  Underspecified  goals:  Goals  may  be  difficult  to  achieve  because  they  are  too  vague, 

5)  Independent  agents:  There  are  independent  entities  in  the  environment  that  influence  it 
(they  may  have  different  goals  than  the  decision  maker). 

We  interviewed  CF  personnel  who  had  experiences  that  were  intuitively  identified  as  being 
complex,  and  then  created  descriptions  of  decision  making  situations  they  had  experienced 
(“scenarios”).  We  then  examined  the  scenarios  to  determine  whether  the  five  complexity  factors 
were  present;  in  all  cases  the  five  factors  were  found.  This  indicates  that  Canadian  Forces 
personnel  do  indeed  experience  the  challenges  identified  by  complexity  research. 

Further  research  also  identified  many  additional  challenges  faced  by  Canadian  Forces  personnel. 
The  broadest  and  most  important  of  these  additional  challenges  have  to  do  with  collaborating 
effectively,  whether  within  their  own  team,  with  other  Canadian  organizations,  or  with  other 
cultures  (e.g.,  Afghan  nationals).  Additional  areas  where  decision  making  could  be  improved  are 
planning  and  dealing  with  resource  challenges.  Future  work  should  be  undertaken  to  examine 
which  of  the  complexity  factors  have  the  most  impact  on  decision  making  in  the  Canadian  Forces, 


Human  systems "  Incorporated 


Page  iii 


HUMAN.Vrsrf  MS 


and  interventions  should  be  aimed  at  increasing  decision  making  proficiency  in  those  areas.  In  the 
scenarios  we  encountered,  personnel  who  were  engaged  in  domestic  and  expeditionary  contingency 
operations  appeared  to  encounter  noticeably  higher  levels  of  complexity  in  their  decision  making 
than  those  involved  in  routine  functions.  This  suggests  that  initial  efforts  to  better  support  complex 
decision  making  should  target  personnel  involved  in  contingency  operations  rather  than  routine 
day-to-day  functions. 
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Mappage  de  la  pertinence  de  la  prise  de  decisions  complexes  pour  les 
operations  terrestres  des  Forces  canadiennes 

Lisa  A.  Rehak,  Tamsen  E.  Taylor  and  Lora  Bruyn  Martin,  Humansyste/ws®  Incorporated; 
DRDC  Toronto  CR2011-079;  R&D  pour  la  defense  Canada  -  Toronto;  mars  2011. 

Les  chercheurs  qualifient  generalement  de  «  complexes  »  les  milieux  decisionnels  difficiles  comme 
ceux  dans  lesquels  les  Forces  canadiennes  (FC)  sont  appelees  a  servir.  11s  ont  avance  que  les 
operations  militaires  modemes  possedent  des  caracteristiques  qui  ont  fait  l’objet  de  vastes  etudes 
dans  des  domaines  comme  la  chimie,  la  physique  et  la  biologie,  dont  les  resultats  pourraient 
s’appliquer  a  l’etude  de  la  prise  de  decisions  dans  les  FC.  Ce  projet  avait  pour  but  premier  de 
determiner  si  l’etude  de  processus  decisionnels  complexes  serait  utile  a  la  prise  de  decisions  qui 
constitue  la  realite  du  personnel  des  Forces  canadiennes,  et  comment  ces  travaux  pourraient  servir  a 
ameliorer  l’education  et  l’instruction  des  militaires  canadiens  en  ce  qui  conceme  la  prise  de 
decisions. 


Les  milieux  decisionnels  difficiles  exigent  une  serie  de  decisions  interdependantes,  dans  un 
contexte  qui  change  a  la  fois  de  fa<;on  autonome  et  en  fonction  des  mesures  que  prend  le  decideur 
et  dont  la  synchronisation  est  primordiale.  Dans  le  cadre  du  projet,  nous  avons  examine  brievement 
la  documentation  sur  la  complexite  et  releve  des  elements  qui  influencent  la  difficulte  du  processus 
decisionnel  dans  des  milieux  complexes.  Le  projet  a  porte  principalement  sur  cinq  de  ces  facteurs 
lies  a  la  complexite,  soit : 

1)  La  connectivite  :  les  facteurs  environnementaux  s’ influencent  mutuellement  entre  eux 
de  maniere  complexe  et  imprevisible; 

2)  La  dynamique  :  le  systeme  comporte  des  aspects  qui  evoluent.  Par  exemple, 
l’environnement  change  avec  le  temps  meme  si  l’on  n’intervient  pas;  le  rythme  auquel 
les  choses  evoluent  peut  varier;  il  peut  y  avoir  un  decalage  entre  les  gestes  et  les 
consequences, 

3)  Buts  conflictuels  multiples  :  il  est  possible  que  l’on  ait  a  atteindre  de  nombreux 
objectifs  qui  ne  sont  pas  tous  realisables  en  meme  temps; 

4)  Buts  imprecis  :  les  buts  peuvent  etre  difficiles  a  atteindre  parce  qu’ils  sont  trop  vagues; 

5)  Agents  independants  :  il  y  a  dans  l’environnement  des  entries  independantes  qui 
l’influencent  (leurs  objectifs  peuvent  etre  differents  de  ceux  du  decideur). 

Nous  avons  interroge  des  membres  du  personnel  des  FC  ayant  vecu  des  experiences  qualifiees 
intuitivement  de  complexes,  puis  nous  avons  elabore  des  descriptions  de  situations  decisionnelles 
qu’ils  avaient  du  affronter  (des  «  scenarios  »).  Nous  avons  ensuite  examine  les  scenarios  afm  de 
determiner  si  les  cinq  facteurs  de  complexite  etaient  presents.  Les  cinq  facteurs  etaient  presents 
dans  tous  les  cas,  ce  qui  indique  que  les  membres  des  Forces  canadiennes  affrontent  effectivement 
les  defis  mentionnes  dans  les  etudes  sur  la  complexite. 
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D’autres  etudes  ont  permis  de  relever  de  nombreuses  autres  difficultes  qui  se  posent  au  personnel 
des  Forces  canadiennes.  Le  plus  repandu  et  le  plus  important  de  ces  defis  est  celui  d’obtenir  une 
collaboration  efficace,  que  ce  soit  au  sein  de  sa  propre  equipe,  avec  d’autres  organisations 
canadiennes,  ou  avec  d’autres  cultures  (p.  ex.,  les  Afghans).  D’autres  secteurs  dans  lesquels  on 
pourrait  ameliorer  la  prise  de  decisions  sont  la  planification  et  les  difficultes  liees  aux  ressources. 
On  devrait  entreprendre  d’autres  etudes  afin  d’examiner  lesquels  des  facteurs  de  complexite  ont  la 
plus  forte  incidence  sur  le  processus  decisionnel  des  Forces  canadiennes,  et  les  interventions 
devraient  etre  axees  sur  la  renforcement  des  competences  decisionnelles.  Comme  les  membres  des 
Forces  canadiennes  qui  sont  engages  dans  des  operations  nationales  et  expeditionnaires  semblent 
etre  appeles  a  prendre  les  decisions  les  plus  complexes,  les  premiers  efforts  d’education  et 
d’ instruction  devrait  probablement  viser  ces  personnes  plutot  que  des  militaires  qui  exercent  des 
fonctions  courantes  a  l’echelle  nationale. 
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This  section  contains  information  about  the  background,  scope,  objectives,  and  deliverables  for  the 
project. 

1.1  Background 

The  Canadian  Forces  (CF)  are  interested  in  improving  the  decision  making  skills  of  commanders 
and  their  staffs  in  contemporary  operations.  The  missions  faced  by  the  CF,  both  present  and  future, 
are  made  challenging  by  many  factors.  Some  of  these  challenges  include  interacting  with  and 
caring  for  the  welfare  of  a  civilian  population  while  dealing  with  a  “non-conventional”  adversary, 
the  need  to  consider  second  and  third-order  effects  of  tactical  actions,  and  the  need  to  interact  with 
other  organizations  and  nations  in  Joint,  Interagency,  Multinational,  and  Public  (JIMP)  operations 
(Godeffoy,  2007). 

Challenging  decision  making  environments  such  as  those  experienced  by  the  CF  are  commonly 
being  characterized  as  “complex”  by  researchers  (e.g.,  Grisogono,  2010;  Moffat,  2003). 
Researchers  have  proposed  that  contemporary  military  operations  exhibit  characteristics  that  have 
been  studied  extensively  in  domains  such  as  chemistry,  physics,  and  biology,  and  research  findings 
in  these  domains  could  be  applied  to  the  study  of  CF  decision  making. 

Although  there  is  some  debate  in  the  literature  concerning  what  makes  decision  making  complex, 
complex  decision  making  typically  involves  decision  making  that  must  take  into  account  many 
interrelated  factors  if  decisions  are  to  have  the  effects  intended  by  a  Decision  Maker  (DM). 

Existing  research  investigating  complex  decision  making  has  frequently  been  conducted  in 
laboratories  using  “microworlds”  (i.e.,  computer  simulations  of  decision  making  environments;  see 
Brehmer,  1992;  Domer  &  Brehmer,  1993).  In  contrast  to  “real-world”  decision  making,  the  use  of 
microworlds  offers  certain  advantages  such  as  more  control  over  the  environment  (e.g.,  specific 
effects  can  be  examined  that  might  not  be  predictable  in  the  real  world;  experimenters  can  increase 
the  speed  of  environmental  change  to  examine  more  effects  in  a  limited  time).  However,  it  is  not 
yet  clear  that  findings  obtained  in  the  laboratory  with  microworlds  are  relevant  to  decision  making 
in  environments  such  as  those  experienced  by  the  CF. 

This  project  is  intended  to  find  similarities  between  the  results  from  existing  research  investigating 
complex  decision  making  and  the  decision  making  performed  by  CF  personnel.  The  main  reason 
for  identifying  these  similarities  is  to  determine  the  extent  to  which  the  existing  research  can  be 
applied  in  order  to  facilitate  and  improve  the  CF  education  and  training  processes  with  respect  to 
decision  making.  Establishing  a  link  between  existing  complex  decision  making  research  findings 
and  the  actual  decision  making  experienced  by  CF  personnel  is  a  logical  and  necessary  first  step 
toward  applying  complex  decision  making  research  to  improving  CF  education  and  training  related 
to  decision  making. 

1.2  Objectives,  Scope,  and  Deliverables 

The  overall  objective  of  this  project  was  to  determine  the  usefulness  of  existing  research  in 
complex  decision  making  (in  particular,  findings  from  research  using  microworlds)  for  enhancing 
military  education  and  training  related  to  decision  making.  To  achieve  this  overall  objective,  first,  a 
brief  literature  review  was  performed  to  determine  the  main  components  of  complex  decision 
making  as  described  in  the  research  literature.  Second,  a  workshop  was  held  with  the  project  team, 
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some  of  whom  had  experience  in  complexity  research  and  some  who  had  military  experience.  The 
objectives  of  the  workshop  were  to  review  the  complexity  components  summarized  in  the  literature 
review,  determine  which  of  the  identified  complexity  components  to  focus  on  during  data 
collection  and  data  analysis,  and  to  determine  an  appropriate  approach  for  data  collection  (e.g., 
who  should  be  interviewed).  Third,  data  were  collected  through  knowledge  elicitation  sessions  with 
SMEs  identified  in  the  workshop  as  having  appropriate  experiences.  Fourth,  several  sets  of 
analyses  were  conducted  to  describe  the  actual  experiences  of  the  CF  personnel  and  to  map  these 
experiences  onto  the  selected  complexity  components.  Finally,  recommendations  and  conclusions 
were  produced  based  on  the  analyses,  and  included  a  discussion  of  whether  the  mapping  was 
sufficient  to  recommend  that  complexity  research  can  be  logically  extended  to  CF  education,  and 
training  as  well  as  to  determine  the  main  challenge  areas  actually  experienced  by  CF  personnel  to 
guide  future  changes  in  the  CF  education  and  training  processes. 

The  scope  of  this  project  was  land  force  focused  (rather  than  focused  on  air  force  or  naval 
operations),  and  included  contingency  operations  (both  domestic  and  expeditionary)  and  day-to- 
day  functions.  It  was  beyond  the  scope  of  this  project  to  validate  the  findings  of  complexity 
research.  Rather,  we  assessed  the  usefulness  of  complexity  research  findings  for  military  decision¬ 
making  education  and  training  based  on  the  assumption  that  the  findings  in  the  existing  research 
literature  were  correct  (although  we  noted  some  cases  in  which  the  research  findings  appeared 
incomplete). 

Deliverables  included: 

1)  A  Mind  Map  containing  a  summary  of  the  characteristics  of  complex  decision  making; 

2)  Descriptions  of  CF  scenarios  which  were  anticipated  to  qualify  as  complex  decision 
making  contexts; 

3)  A  mapping  of  selected  main  ideas  in  complexity  theory  to  the  scenario  descriptions; 

4)  A  ranking  of  the  relative  contribution  of  the  main  complexity  ideas  to  the  overall 
challenges  present  in  the  scenario; 

5)  A  summary  of  the  main  challenges  present  in  the  scenarios;  and 

6)  Possible  education  and  training  interventions/enhancements. 

1.3  Outline  of  report 

The  first  chapter  of  this  report  contains  a  brief  background  and  discussion  of  the  objectives,  scope, 
and  deliverables.  The  second  chapter  describes  the  methodology  used  for  the  literature  review,  the 
workshop,  the  knowledge  elicitation,  and  the  analyses.  The  third  chapter  describes  the  results  of  the 
literature  review,  workshop,  and  analyses,  and  the  fourth  chapter  contains  the  conclusions  and 
recommendations  suggested  by  the  analyses. 
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2.  Methodology 


In  this  section,  we  review  the  methodologies  used  for  the  literature  review,  the  workshop,  the 
knowledge  elicitation  sessions,  as  well  as  the  processes  followed  during  the  data  analysis.  Results 
are  not  included  here,  but  can  be  found  in  next  section  (Section  3). 

2.1  Review  of  complexity  literature 

This  was  not  intended  to  be  a  complete  review  of  the  complexity  literature,  but  was  instead  a 
guided  review  to  gain  an  overview  of  the  characteristics  of  complex  decision  making  and  factors 
that  were  thought  to  make  decision  making  complex.  The  Scientific  Authority  (SA)  provided 
several  documents  to  review,  including: 

•  Brehmer,  B.  &  Dorner,  D.  (1993).  Experiments  with  computer-simulated  microworlds: 
Escaping  both  the  narrow  straits  of  the  laboratory  and  the  deep  blue  sea  of  the  field  study. 
Computers  in  Human  Behavior,  9,  171-184. 

•  Department  of  Defence  (2009).  Adaptive  campaigning  09:  Army’s  future  land  operating 
concept,  Australian  Army,  Canberra. 

•  Dorner,  D.  (1996).  The  logic  of  failure.  New  York,  Metropolitan  Books. 

•  Funke,  J.  (2001).  Dynamic  systems  as  tools  for  analysing  human  judgement.  Thinking  and 
Reasoning,  7,  69-89. 

•  Grisogono,  A.  M.  (2010).  Overview  of  Complex  Decision-Making  (CxDM)  program 
(draft). 

•  Grisogono,  A.  M.  (2006).  The  implications  of  complex  adaptive  systems  theory  for  C2. 
Paper  presented  at  CCRTS:  The  State  of  the  Art  and  the  State  of  the  Practice. 

•  Moffat,  J.  (2003).  Complexity  theory  and  network  centric  warfare.  CCRP  Publication 
Series,  available  for  download  at  www.dodccrp.org. 

Previous  research  conducted  by  HSI®  which  investigated  dynamic  decision  making  was  also 
reviewed  (Brown,  Karthaus,  Rehak,  &  Adams,  2009).  It  is  unclear  what  if  anything  separates 
complex  decision  making  from  dynamic  decision  making,  as  they  share  many  of  the  same 
characteristics  (e.g.,  they  both  involve  interrelated  decisions).  That  is,  dynamic  decision  making 
and  complex  decision  making  appear  to  be  very  similar  phenomena  called  two  different  things  by 
different  sets  of  researchers.  For  the  remainder  of  this  report  we  will  refer  to  the  type  of  decision 
making  we  are  focussing  on  as  complex,  although  we  will  still  use  some  of  the  findings  from  the 
dynamic  decision  making  literature  for  background. 

The  definitions  of  complex  decision  making  are  varied  and  complex  themselves.  The  approach 
taken  to  define  complex  decision  making  was  to  compile  a  list  of  components  that  can  be  found  in 
the  complex  decision  making  literature,  and  see  which  components  separate  complex  decisions 
from  decisions  which  are  difficult  for  other  reasons  (i.e.,  “complex”  vs.  “complicated”  decision 
making).  This  factor  list  was  then  presented  in  mind  map  format  and  used  as  a  basis  for  discussion 
in  the  workshop  (see  Section  3.1  for  a  detailed  description  of  the  results  of  the  literature  review). 
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2.2  Workshop 

The  goals  of  the  workshop  were  to  review  the  findings  of  the  literature  review,  and  to  finalize  the 
scope  of  the  project,  particularly  with  respect  to  the  complexity  concepts  that  would  be  focused  on 
in  the  knowledge  elicitation  sessions.  Specifically,  more  information  was  required  about  which 
aspects  of  complexity  to  focus  on  (as  it  was  unlikely  that  they  could  all  be  covered  easily  in  each 
interview)  and  to  determine  which  CF  environments  and  decision  making  contexts  were  of  major 
interest  so  that  SMEs  could  be  recruited  to  be  interviewed  for  the  project. 

The  workshop  took  place  on  Thursday,  August  26,  2010,  at  Defence  Research  and  Development 
Canada  (DRDC)  -  Toronto,  from  approximately  10am  to  3pm.  The  workshop  was  attended  by  the 
project  team,  including  the  Scientific  Authority  and  the  HSI®  project  team.  As  well,  a  few  other 
researchers  from  DRDC  Toronto  attended  part  of  the  workshop  as  they  were  available. 

2.2.1  Prioritization  and  definition  of  complexity  factors 

The  mind  map  produced  from  the  literature  review  (see  Annex  A)  was  reviewed  by  the  workshop 
participants,  and  minor  edits  were  suggested. 

Once  the  results  from  the  literature  review  had  been  agreed  upon  a  discussion  was  held  to 
determine  which  of  the  complexity  factors  identified  in  the  literature  review  were  of  primary 
interest.  Once  main  complexity  factors  were  selected  (Connectivity,  Dynamics,  Multiple 
conflicting  goals,  Under-specified  goals,  and  Independent  agents  -  a  full  description  can  be  found 
in  Section  3.2),  the  definitions  of  those  factors  were  re-examined  to  ensure  agreement. 

2.2.2  Potential  scenarios  and  SMEs 

Once  primary  complexity  factors  had  been  selected  for  the  knowledge  elicitation  sessions,  potential 
CF  operation  types  were  discussed  to  determine  overall  areas  which  should  be  considered  in  scope 
(e.g.,  were  we  interested  in  domestic  operations,  expeditionary  operations,  or  should  both  be  in 
scope?).  Once  these  main  types  of  CF  operations  were  discussed,  specific  examples  that  would 
likely  contain  the  complexity  factors  of  primary  interest  were  listed,  and  possible  Points  of  Contact 
(PoCs)  were  identified.  Based  on  who  had  previous  knowledge  of  the  potential  SMEs  or  PoCs, 
workshop  members  were  assigned  to  follow  up  with  the  identified  SMEs  and  PoCs  to  recruit 
participants  (full  description  of  scenarios  and  SMEs  can  be  found  in  Section  3.2.2). 

2.3  Knowledge  elicitation 

The  goal  of  the  knowledge  elicitation  sessions  was  to  obtain  information  about  the  decision  making 
actually  experienced  by  CF  personnel,  both  to  allow  the  creation  of  descriptions  of  those 
experiences  and  to  facilitate  a  comparison  between  the  selected  complex  decision  making  factors 
and  those  CF  experiences. 

Knowledge  elicitation  involved  interviewing  SMEs  who  had  experience  in  the  decision  making 
contexts  intuitively  identified  during  the  workshop  as  complex.  The  interview  technique  used  was 
semi-structured,  individual  interviews  based  on  the  Critical  Decision  Method  (CDM)  technique. 
The  interviews  were  performed  according  to  DRDC  ethics  guidelines,  under  Protocol  L-763 
(Jarmasz,  J.,  Rehak,  L.,  Taylor,  T.,  Bruyn  Martin,  L.,  Karthaus,  C.,  2011).  The  phases  of  the 
interviews  are  described  in  the  sections  that  follow. 
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2.3.1  Phase  1:  Introduction  of  interview  participants  and  demographics 

This  phase  consisted  of  introducing  the  interviewers  to  the  SME  and  vice  versa,  to  facilitate  the 
discussion.  Each  SME  was  asked  to  answer  demographic  questions  at  the  beginning  of  the 
interview  to  provide  a  record  of  their  experience.  Demographic  questions  which  were  asked 
included1: 

•  What  is  your  rank? 

•  How  long  have  you  been  in  the  military? 

•  What  is  your  current  position  ? 

•  How  long  have  you  held  your  current  position  ? 

•  What  other  positions  have  you  held? 

In  general,  the  decision-making  context  that  had  previously  been  identified  was  reviewed  during 
the  discussion  of  demographic  details  to  ensure  that  the  SME  actually  had  the  anticipated 
experience,  and  that  they  understood  the  decision  making  context  that  we  thought  was  complex.  In 
most  cases,  the  interviewer  also  indicated  that  other  decision  making  contexts  would  be  considered 
for  further  discussion  if  the  SME  thought  that  another  of  their  experiences  was  more  complex  than 
the  one  we  identified. 

2.3.2  Phase  2:  Project  introduction  and  background 

Phase  2  introduced  the  project  background  and  goals  to  the  SME.  The  SME  had  some  level  of 
knowledge  about  the  project  and  its  goals  which  he  or  she  was  given  when  they  were  recruited  to 
participate.  However,  due  to  the  time  which  elapsed  since  the  SME  agreed  to  be  interviewed,  it  was 
considered  beneficial  to  refresh  his  or  her  memory.  As  well,  this  face-to-face  discussion  of  the 
project  purpose  allowed  the  SME  to  ask  the  interviewers  questions  about  the  project  in  general 
before  he  or  she  began  to  answer  questions. 

The  specific  goals  of  the  project  were  stated,  something  similar  to: 

This  project  is  intended  to  find  similarities  between  the  results  of  existing  laboratory-based 
research  in  complex,  dynamic  decision  making  and  the  decision  making  performed  by  CF 
personnel.  The  main  advantage  of  identifying  these  similarities  would  be  to  extend  the  research 
results  in  order  to  facilitate  and  improve  the  CF  educational  process. 

It  should  be  noted  that  if  the  SME  had  questions  about  the  nature  of  complex  decision  making  at 
this  time,  we  mentioned  that  we  would  shortly  be  discussing  five  main  factors  involved  in  complex 
decision  making  which  would  give  the  SME  a  clearer  understanding  of  what  we  meant  by  the  term. 

After  the  SME  indicated  that  he  or  she  understood  the  general  reason  for  the  interview,  and  any 
questions  that  the  SME  had  had  been  answered,  a  brief  overview  of  the  anticipated  interview 
timeline  was  provided: 

•  Demographic  questions  and  general  introduction  (intro  and  Phase  1):  5-10  minutes 

•  Project  context  and  key  complexity  factors  (Phase  2):  15  minutes 

•  Scenario  selection  and  general  scenario  details  (Phase  3,  Step  1):  15  minutes 


1  Note  that  in  this  and  following  sections,  italicized  text  indicates  things  that  were  actually  said  to  the  SME. 
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•  Scenario  timelines  and  critical  decisions  (Phase  3,  Step  2):  15  minutes 

•  Scenario  challenges  and  errors  (Phase  3,  Step  3):  30  minutes 

•  Scenario  strategies  and  learning  (Phase  3,  Step  4):  30  minutes 

After  the  timeline  was  discussed  with  the  SME,  the  interviewer  introduced  the  five  main 
complexity  factors  and  defined  them  one  at  a  time,  and  asked  the  SME  to  provide  examples  from 
his  or  her  own  experience  that  seemed  to  show  the  factors.  The  key  complex  decision  making 
factors  discussed  with  the  SME,  along  with  the  definitions  provided,  were  as  follows: 

Key  factors  related  to  complexity  that  we  are  looking  for  include: 

•  Connectivity:  Things  in  the  environment  influence  one  another  in  complicated  and 
unexpected  ways 

•  Dynamics:  The  system  has  aspects  that  unfold  over  time.  For  example,  the  environment 
changes  over  time  even  when  you  do  nothing;  the  rate  at  which  things  change  may  be 
variable;  there  may  be  delays  between  actions  and  effects 

•  Multiple  conflicting  goals:  Having  to  achieve  multiple  objectives  which  may  not  be  all 
achievable  at  the  same  time 

•  Under-specified  goals:  Goals  may  be  difficult  to  achieve  because  they  are  too  vague 

•  Independent  agents:  There  are  independent  entities  in  the  environment  who  influence  it 
(they  may  have  different  goals  than  the  decision  maker) 

Once  the  discussion  of  the  key  factors  was  complete,  the  SME  was  handed  a  sheet  containing  the 
factors  and  their  definitions  (the  material  in  italics  above),  and  the  interview  proceeded  onto  Phase 
3. 

2.3.3  Phase  3:  Description  and  discussion  of  scenario 

The  majority  of  the  data  was  collected  during  this  phase.  The  goal  was  to  obtain  enough 
information  about  a  specific  complex  decision  making  context  experienced  by  the  SME  to  create  a 
scenario  description  and  map  the  five  key  complexity  factors  onto  the  scenario  description  as 
closely  as  possible. 

The  interview  format  was  a  modified  CDM  technique.  This  technique  has  proven  to  be  effective  in 
drawing  out  details  of  SME’s  experiences  related  to  decision  making.  The  components  of  the  CDM 
as  we  used  them  are  as  follows: 

Step  1:  Scenarios  to  explore  further  were  identified.  At  this  point,  a  brief  discussion  was  held  with 
the  SME  to  determine  which  of  the  potential  decisions  they  discussed  earlier  was  the  most 
promising  to  use  as  a  complex  decision  making  scenario.  In  general,  there  were  usually  several 
options,  and  the  one  that  appeared  to  contain  as  many  of  the  five  key  complexity  factors  as  possible 
was  chosen  and  discussed  further. 

As  an  example  context,  the  following  was  developed  related  to  the  Liaison  Officer,  Afghanistan 
interview,  with  the  assumption  that  the  SME  being  interviewed  was  deployed  to  HQ  Afghanistan 
as  a  liaison  officer: 

You  have  been  deployed  to  HQ  Afghanistan  as  the  Canadian  Commander’s  Liaison  Officer.  The 
situation  in  Afghanistan  is  complex,  and  included  issues  such  as  determining  the  validity  of 
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information  and  making  decisions  about  what  to  recommend.  This  was  a  complex  problem  about 
how  to  decide  what  needs  to  be  done. 

•  Can  you  begin  to  list  out  the  factors  increasing  the  complexity  of  the  situation?  Specific 
issues  may  include  (to  be  used  as  examples  if  the  DM  has  difficulty): 

o  How  to  determine  what  factors  should  be  given  most  weight  when  making 
recommendations 

o  How  to  determine  whom  to  trust  and  what  information  to  trust 
o  How  to  communicate  effectively  with  members  of  a  different  culture 
o  How  to  determine  mentoring  needs 

o  How  to  determine  which  activities  best  support  achievement  of  Canada 's  goals  in 
theatre. 

•  Can  you  describe  a  specific  scenario  that  took  place  over  weeks  or  months  that  you  found 
challenging? 

•  What  was  your  role  during  this  scenario? 

•  At  what  point  did  you  become  involved  in  the  context  under  discussion  (e.g.,  acting  as  a 
liaison  officer)? 

•  What  was  your  decision  making  authority  in  this  scenario? 

•  What  factors  made  this  particular  scenario  challenging? 

•  Who  were  the  original  stakeholders  in  this  scenario,  and  did  other  stakeholders  emerge 
over  time? 

•  Did  the  stakeholders  agree  about  what  was  important? 

•  Did  the  stakeholders  influence  the  scenario?  Were  these  influences  anticipated? 

•  What  was  your  perception  of  what  was  involved  in  this  situation  originally?  D  id  your 
perception  of  what  was  involved  in  this  situation  change  over  time?  (E.g.,  were  there  a  set 
of factors  you  knew  were  involved  when  you  first  started,  and  then  you  realized  that 
additional  factors  were  at  play  later  on?) 

•  During  this  scenario,  were  there  times  when  your  decisions  would  have  no  effects  (i.e. 
there  appeared  to  be  strong  factors  maintaining  status  quo)?  Or  a  time  when  your  actions 
would  start  to  have  an  effect,  but  then  the  scenario  returned  back  to  status  quo? 

•  Can  you  think  of  any  examples  where  there  was  a  long  trend  of  no  change,  and  then 
suddenly  there  was  a  dramatic  increase  or  decrease  in  some  scenario  aspect?  ("aspect" 
here  could  be  thought  of  as  higher  level  interest,  change  in  funding,  change  in  manning, 
etc.) 

•  Do  you  have  any  examples  of  a  2  sided  escalation  where  Blue  Forces  do  X,  Red  Forces  do 
Y;  Blue  Forces  increase  X,  Red  Forces  increase  Y  continuously? 

•  How  you  ever  had  a  scenario  where  the  same  action  had  different  effects?  Difference 
effects  may  be  immediate,  second,  or  third-order,  etc. 
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•  Or  when  there  were  delays  between  your  actions  and  the  desired  effects?  [Note:  Some 
further  questioning  may  be  desired  to  verify  the  link  between  the  action  and  the  effect -for 
example,  what  if  the  action  actually  had  no  bearing  on  the  desired  effect,  but  the  desired 
effect  was  achieved  regardless?) 

•  Any  examples  of  actions  having  the  opposite  effect  than  what  was  intended  or  anticipated? 

Once  a  more  detailed  scenario  was  agreed  upon,  the  interviewers  followed  up  with  questions 
intended  to  uncover  critical  points  and  additional  desired  information  in  Step  2. 

Step  2:  Once  a  detailed  scenario  was  agreed  upon,  the  goals,  decisions  that  are  made,  information 
used,  and  situational  features  that  affected  performance  during  the  scenario  were  identified  and  the 
timeline  of  critical  points  was  verified.  The  goal  of  this  step  was  to  identify  all  the  critical  points  so 
complexity  behaviours  around  those  critical  points  could  be  focused  on  during  Step  3.  Questions 
similar  to  the  following  were  asked: 

•  What  were  the  critical  points  which  happened  during  this  scenario  (in  sequence)? 

•  At  each  critical  point,  what  decision(s)  or  assessment(s)  did  you  have  to  make? 

The  interviewers  reviewed  the  scenario  and  each  critical  point  that  occurred  during  the  decision 
situation  with  the  interviewee  to  ensure  a  common  understanding.  The  procedure  in  this  step  varied 
depending  on  the  thought  processes  of  the  SME.  Sometimes  the  SME  preferred  to  review  all  of  the 
critical  points  and  then  think  about  related  decisions,  and  sometimes  the  SME  moved  back  and 
forth  between  critical  points  and  decisions.  Elowever  the  SME  seemed  to  wish  to  proceed  was 
accommodated  by  the  interviewer. 

During  this  portion  of  the  interview,  a  timeline  was  created  on  a  blank  sheet  of  paper  to  act  as  a 
memory  cue  for  both  the  SME  and  the  interviewers.  Critical  points  and  decisions  were  written  on 
post-it  notes  and  added  to  the  timeline.  The  use  of  post-it  notes  enabled  order  changes  and  spacing 
changes  along  the  timeline  as  required. 

Once  the  SME  indicated  that  the  critical  points  and  related  decisions  had  been  noted  satisfactorily, 
the  interviewer  moved  on  to  Step  3. 

Step  3 :  Issues  which  needed  to  be  elaborated  were  discussed.  The  main  pieces  of  information  we 
wished  to  get  out  of  this  step  included  challenges  and  potential  errors. 

Let ’s  focus  on  the  decisions  made  at  the  first  critical  point... 

•  What  made  the  decision  challenging? 

•  Were  your  goals  understood? 

•  Did  you  have  to  achieve  more  than  one  goal  at  a  time? 

•  Did  you  have  a  clear  understanding  or  mental  model  of  the  issues  involved? 

•  Did  you  consult  others  for  their  input? 

•  What  did  you  NOT  know  that  you  really  needed  to  know? 

o  Was  there  any  way  you  could  you  have  gotten  that  information? 

•  What  else  (besides  more  information  from  last  question)  would  have  made  this  decision 
easier  for  you? 
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•  Were  you  able  to  think  through  the  2nd  and  3ld  order  effects  of  your  decision  options? 

•  Were  there  any  unanticipated  consequences  of  your  decision? 

•  If  you  could  make  this  decision  again,  what  would  you  do  differently? 

•  Did  you  modify  your  approach  to  decision  making  (e.g.,  level  of  risk  aversion)? 

Each  critical  point  and  related  decision  was  discussed  using  the  questions  above.  After  each 
decision  had  been  reviewed,  the  interviewers  moved  on  to  Step  4. 

Step  4:  During  this  step,  alternatives  were  identified  and  elaborated.  Different  strategies  for 
performing  tasks  were  discussed  and  factors  that  influenced  the  choice  and  usefulness  of  strategies 
were  identified.  Ways  in  which  errors  might  have  been  prevented  were  discussed. 

•  What  were  important  factors  that  you  feel  affected  your  performance  during  the  scenario? 

•  Was  there  a  time  during  this  or  a  similar  scenario  that  someone  did  something  and  you 
would  have  acted  differently? 

•  Did  you  get  feedback  about  how  your  actions  were  affecting  the  scenario? 

•  What  would  have  made  this  scenario  easier  for  you? 

•  Are  there  any  other  types  of  behaviours  that  should  be  taught  to  people  who  are  likely  to 
experience  similar  types  of  scenarios  and  decisions? 

•  What  did  you  learn  from  this  scenario  that  your  training  didn ’t  teach  you? 

•  Did  the  decisions  made  in  this  scenario  follow  the  mission  command  philosophy  -  if  not, 
why  not? 

Near  the  end  of  the  interview,  SMEs  were  explicitly  asked  if  there  were  any  ideas  they  had  about 
how  education  or  training  could  be  improved  so  that  other  people  would  be  better  prepared  to  deal 
with  the  challenges  they  faced  in  this  scenario,  or  if  they  had  any  other  education  or  training 
suggestions. 

2.4  Analysis  methodology 

There  were  several  analyses  performed  on  the  data  acquired  in  the  knowledge  elicitation  sessions. 
The  first  was  a  description  of  the  scenario,  involving  the  creation  of  a  description  of  the  complex 
decision  making  situation  described  by  each  SME.  The  second  analysis  involved  a  mapping  of  the 
five  complexity  factors  to  each  scenario,  with  a  rating  of  the  relative  contribution  of  the  five  key 
complexity  factors  to  the  general  difficulty  of  the  scenario.  The  third  analysis  was  a  compilation  of 
the  main  decision  making  challenges  experienced  by  the  SMEs. 

2.4.1  Scenarios 

The  scenarios  were  created  to  provide  a  description  of  complex  decision  making  situations  actually 
experienced  by  CF  personnel.  For  the  purposes  of  this  project  they  serve  to  illustrate  CF 
experiences  and  they  provide  a  basis  for  determining  whether  the  key  complexity  factors  identified 
from  microworld  research  are  relevant  to  the  actual  experiences  of  CF  personnel.  In  future  work,  it 
may  be  that  these  scenarios  can  be  used  to  facilitate  education  and  training  in  complex  decision 
making  in  the  CF. 
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The  information  gathered  from  the  interviews  was  examined  to  create  a  summary  of  the 
information  provided  by  the  SMEs.  The  scenarios  contain  a  general  description  of  the  role  of  the 
DM,  and  a  list  of  the  factors  that  were  mentioned  by  the  DM  as  playing  a  large  role  in  his  or  her 
decision  making.  Factors  that  unfolded  over  time  were  pointed  out  as  dynamic  events  are 
fundamental  to  complex  decision  making.  Each  scenario  was  submitted  to  the  appropriate  SME  for 
review,  and  scenarios  were  revised  according  to  the  SMEs’  suggestions.  The  scenarios  were  as 
descriptive  as  possible,  although  the  individual  SME  is  not  named  in  them.  The  SMEs  removed  any 
identifying  details  that  they  did  not  wish  to  be  included. 

2.4.2  Mapping  of  five  complexity  factors  to  scenarios 

One  of  the  main  goals  of  this  project  was  to  determine  whether  five  main  factors  identified  from 
the  complexity  literature  can  be  mapped  to  the  operational  experiences  of  members  of  the  CF.  To 
facilitate  this,  an  explicit  mapping  from  the  scenarios  to  the  five  key  complexity  factors  was 
performed.  These  are  presented  in  tabular  format  using  the  following  template: 


Table  1:  Template  for  complexity  factor  mapping  to  scenarios 


Complexity  Factor  and  Definition 

Factor  Examples 

Importance  of  Factor  for 
Scenario  (High,  Medium, 
Low)  and  Justification 

Connectivity: 

Things  in  the  environment  influence  one  another 
in  complicated  and  unpredictable  ways 

Examples  of  decisions  which  involved 
interrelated  factors  included: 

Dynamics: 

The  system  has  aspects  that  unfold  over  time. 

For  example,  the  environment  changes  over  time 
even  when  you  do  nothing;  the  rate  at  which 
things  change  may  be  variable;  there  may  be 
delays  between  actions  and  effects. 

It  is  important  to  note  hierarchical  aspects  if 
present  (i.e.,  subsystems  that  have  their  own 
dynamics  which  are  part  of  the  DM  context). 

Situations  that  unfolded  over  time  that 
profoundly  affected  decision  making 
included: 

There  were  subsystems  which  had 
their  own  dynamics  in  this  context. 
These  included: 

Multiple  conflicting  goals: 

Having  to  achieve  multiple  objectives  which  may 
not  be  all  achievable  at  the  same  time 

The  goals  of  the  DM  included: 

Under-specified  goals: 

Goals  may  be  difficult  to  achieve  because  they 
are  too  vague 

Examples  of  underspecified  goals 
included: 

Independent  agents: 

There  are  independent  entities  in  the 
environment  who  influence  it  (they  may  have 
different  goals  than  the  decision  maker) 

Independent  agents  who  influenced 
decision  making  included: 

Because  it  rapidly  became  obvious  that  examples  of  the  complexity  factors  could  be  found  in  the 
scenarios,  a  more  quantitative  approach  was  attempted  to  determine  the  relative  importance  of  the 
different  complexity  factors  to  each  of  the  scenarios.  That  is,  rather  than  just  a  yes/no  assessment 
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that  the  complexity  factor  was  or  was  not  present  in  the  scenario,  it  was  desirable  that  an 
assessment  be  made  about  which  complexity  factors  were  primarily  responsible  for  the  complexity 
of  a  particular  decision  making  scenario.  Because  there  were  not  resources  available  to  create  an 
objective  complexity  rating  system,  the  rating  system  was  relative  and  only  considered  the  extent  to 
which  the  complexity  factors  were  present  to  different  degrees  in  the  final  10  scenarios  created  for 
this  project.  That  is,  a  complexity  factor  was  rated  as  HIGH  for  a  scenario  if  it  was  present  in  that 
scenario  more  than  in  others,  and  LOW  for  a  scenario  if  it  was  present  less  in  that  scenario 
compared  to  the  other  9.  It  should  be  noted  that  these  comparisons  can  only  take  into  account  the 
information  provided  by  the  SMEs  about  their  experiences,  and  it  is  likely  that  the  relative  ratings 
are  profoundly  affected  by  the  SMEs’  own  interpretations  and  understanding  of  their  experience,  as 
well  as  their  ability  to  communicate  to  the  interviewers. 

There  were  several  steps  used  to  create  a  rating  for  the  five  complexity  factors.  First,  general 
criteria  were  created  based  on  the  definitions  of  the  five  complexity  factors  examined  in  the 
project.  For  example,  the  Connectivity  factor  was  defined  as  “Things  in  the  environment  influence 
one  another  in  complicated  and  unpredictable  ways”.  From  this,  two  criteria  were  created,  one 
being  “How  many  interrelated  factors  have  to  be  considered  when  making  decisions?”  and  the 
other  was  “How  many  2nd  and  3rd  order  effects  were  noted?” 

Once  these  general  criteria  were  determined,  each  of  the  10  scenarios  was  examined  to  assess  the 
extent  to  which  the  general  criteria  applied  to  that  scenario.  For  example,  the  scenarios  were 
examined  to  determine  the  maximum  number  of  interrelated  factors  that  had  to  be  considered  when 
making  a  single  decision.  Once  these  assessments  were  made,  they  were  compiled  and  compared  to 
determine  what  should  be  considered  as  “HIGH”,  “MEDIUM”,  and  “FOW”  ratings  for  each  of  the 
criteria.  This  assessment  was  based  on  the  overall  range  present  across  the  scenarios  (e.g.,  all  of  the 
counts  of  maximum  number  of  interrelated  factors  related  to  a  single  decision  were  compiled).  As 
well,  an  attempt  was  made  to  put  approximately  equal  numbers  of  scenarios  into  the  three  rating 
categories  (i.e.,  HIGH,  MEDIUM,  and  FOW)  so  that  there  would  be  a  range  of  ratings  and 
differences  could  be  detected  (e.g.,  avoid  floor  or  ceiling  effects). 

For  the  template  used  to  create  the  complexity  rating  criteria,  see  Table  2:  Template  for  creating 
complexity  rating  criteria.  Note  that  all  “General  Criteria”  (found  in  the  middle  column  of  Table  2 
below)  were  centred  on  the  decision  maker  (e.g.,  “Are  there  many  goals?”  is  asking  whether  the 
decision  maker  had  many  goals). 


Table  2:  Template  for  creating  complexity  rating  criteria 


Factor  and  Definition 

General  Criteria 

Rating  (High,  Medium,  Low)  and 
Justification 

Connectivity: 

Things  in  the  environment 
influence  one  another  in 
complicated  and  unpredictable 
ways 

How  many  interrelated  factors  have 
to  be  considered  when  making 
decisions? 

How  many  2nd  and  3rd  order  effects 
were  noted? 

High 

<Criteria> 

Medium 

<Criteria> 

Low 

<Criteria> 

Dynamics: 

Were  there  examples  of  situations 

High 
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Factor  and  Definition 

General  Criteria 

Rating  (High,  Medium,  Low)  and 
Justification 

The  system  has  aspects  that 
unfold  over  time.  For  example,  the 
environment  changes  over  time 
even  when  you  do  nothing;  the 
rate  at  which  things  change  may 
be  variable;  there  may  be  delays 
between  actions  and  effects. 

It  is  important  to  note  hierarchical 
aspects  if  present  (i.e., 
subsystems  that  have  their  own 
dynamics  which  are  part  of  the 

DM  context). 

that  unfolded  over  time  that 
profoundly  affected  decision 
making? 

How  many  subsystems  impacted 
the  decision-making  context? 

Were  there  subsystems  that  had 
hierarchical  aspects  (e.g.,  their  own 
dynamics)? 

<Criteria> 

Medium 

<Criteria> 

Low 

<Criteria> 

Multiple  conflicting  goals: 

Having  to  achieve  multiple 
objectives  which  may  not  be  all 
achievable  at  the  same  time 

Are  there  many  goals? 

Are  there  problems  with  goal 
prioritization? 

Do  these  goals  necessarily  conflict? 

Are  there  situations  where  the  goals 
conflict  even  if  they  wouldn’t 
necessarily  have  to? 

High 

<Criteria> 

Medium 

<Criteria> 

Low 

<Criteria> 

Under-specified  goals: 

Goals  may  be  difficult  to  achieve 
because  they  are  too  vague 

Are  there  goals  that  are  vague? 

To  what  extent  did  the  vagueness  of 
goals  complicate  decision  making? 

High 

<Criteria> 

Medium 

<Criteria> 

Low 

<Criteria> 

Independent  agents: 

There  are  independent  entities  in 
the  environment  who  influence  it 
(they  may  have  different  goals 
than  the  decision  maker) 

Were  there  independent  agents  who 
impact  decision  making? 

Did  independent  agents  have  goals 
that  conflicted  with  the  DM’s  goals? 
Did  independent  agents  have  goals 
that  conflicted  with  one  another  in 
such  a  way  that  the  DM’s  decision 
making  was  affected? 

High 

<Criteria> 

Medium 

<Criteria> 

Low 

<Criteria> 

Once  these  ratings  were  complete,  they  provided  an  overview  of  the  relative  complexity  of  the  10 
scenarios.  These  ratings  were  also  used  in  the  top-down  challenges  analysis  (the  full  list  of  ratings 
is  provided  in  Section  3.4). 

2.4.3  Challenges  analysis 

The  goal  of  the  challenges  analysis  was  to  determine  the  types  of  challenges  experienced  by  CF 
personnel  in  operations,  with  the  goal  of  guiding  future  CF  education  and  training  related  to 
complex  decision  making.  There  were  two  components  to  this  analysis:  one  approach  was  top- 
down,  the  other  was  bottom-up.  The  top-down  challenges  analysis  examined  the  challenges 
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identified  via  the  mapping  from  the  complexity  literature  to  the  scenarios,  and  the  ratings  used  to 
assess  the  relative  contribution  of  the  five  main  complexity  factors  to  the  difficulty  of  each 
scenario.  That  is,  this  approach  was  top-down  as  it  was  guided  by  previously  identified  factors  that 
were  expected  to  play  a  role  in  making  decision-making  complex.  The  bottom-up  challenges 
analysis  examined  the  decision  making  challenges  mentioned  by  the  SMEs  during  their  interviews, 
and  examined  them  to  determine  if  there  were  any  categories  or  themes  that  emerged  which 
represent  broad  challenge  areas  that  could  be  addressed  in  CF  education.  That  is,  this  approach  was 
bottom-up  as  it  was  driven  by  the  data  provided  by  the  SMEs  during  the  interviews,  rather  than 
preconceived  ideas  about  what  would  make  decision  making  challenging.  The  results  from  these 
two  types  of  identified  challenges  were  then  compared  to  determine  whether  the  complexity 
literature  seemed  to  provide  an  adequate  understanding  of  the  difficulties  faced  by  CF  personnel 
when  they  are  making  decisions  during  operations  . 

Another  goal  was  to  determine  if  different  types  of  challenges  were  present  in  different  types  of 
scenarios.  Challenge  ratings  and  challenge  categories  were  examined  across  scenario  type  to 
determine  if  there  was  a  distinguishable  pattern  of  challenges.  The  idea  was  that,  if  different  types 
of  challenges  occur  in  different  types  of  scenarios,  then  specialization  in  decision  making 
instruction  might  be  required  so  that  personnel  are  educated  and  trained  to  deal  with  the  challenges 
that  they  are  likely  to  face. 

2. 4.3.1  Top-down  challenges  analysis 

The  top-down  challenges  analysis  was  based  upon  the  ratings  created  for  the  mapping  of 
complexity  factors  onto  the  scenarios.  See  Section  2.4.2  for  a  description  of  the  procedure  used  to 
create  these  ratings.  Another  important  question  that  can  be  addressed  by  this  analysis  is  whether 
different  scenario  types  appeared  to  differ  on  their  overall  level  of  complexity  or  complexity 
components.  To  determine  this,  the  complexity  of  the  scenarios  were  scored  for  each  factor 
(“E1IGE1”  =  2,  “MEDIUM”  =  1,  and  “LOW”  =  0),  the  scores  were  totalled  to  get  an  overall 
complexity  score,  and  then  the  scenarios  were  ordered  based  on  their  overall  complexity  and 
patterns  were  noted  (e.g.,  what  types  of  scenarios  appeared  to  be  highest  in  complexity).  Each 
complexity  factor  was  examined  in  the  same  way  to  determine  if  there  were  patterns  in  the  degree 
to  which  each  factor  manifested  in  the  different  types  of  scenarios. 

Once  it  was  determined  which  scenarios  rated  as  “HIGH”  on  different  complexity  factors,  they 
were  examined  to  determine  if  there  were  any  common  characteristics  which  might  distinguish 
scenarios  which  were  likely  to  contain  that  particular  complexity  factor. 

2. 4.3. 2  Bottom-up  challenges  analysis 

The  bottom-up  challenges  analysis  was  based  upon  things  that  the  SMEs  explicitly  mentioned  as 
being  challenging  about  the  decisions  they  had  to  make  in  the  scenario  they  were  describing.  The 
information  gathered  during  the  knowledge  elicitation  sessions  was  examined  and  challenges  were 
compiled  from  them.  Also  included  in  this  list  are  specific  education  and  training  suggestions 
offered  by  the  SMEs.  The  list  of  challenges  and  education  and  training  suggestions  were  compiled 
into  a  spreadsheet  using  the  format  presented  in  Table  3.  Once  the  list  was  compiled,  the  challenges 
were  examined  to  determine  if  there  were  broader  themes  and  subthemes  present.  These  themes 
were  then  summarized  in  the  table.  Note  that  descriptions  of  the  general  themes  are  provided  in  the 
themes  analysis  (e.g.,  see  Table  5  in  Section  2.4.4. 1). 
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Table  3:  Bottom-up  challenges  template 


Scenario 

Recommendation 

Challenge  or 
Education/Training? 

General  theme 

Subtheme 

Once  these  analyses  were  performed,  they  were  compared  and  examined  to  form  conclusions  to 
support  education  and  training  initiatives. 

2.4. 3. 3  Bottom-up  education  and  training  suggestions 

Because  one  of  the  key  goals  of  this  project  is  to  support  CF  education  and  training,  once  the 
bottom-up  challenge  analysis  was  complete,  the  list  was  examined  and  the  number  of  education 
and  training  suggestions  for  each  general  theme  was  identified.  The  specific  suggestions  were 
examined,  and  general  suggestions  were  derived  from  them.  These  general  suggestions  were 
entered  in  the  template  presented  in  Table  4. 


Table  4:  Bottom-up  education  and  training  suggestions  template 


General 

Theme 

Description 

General  Education/Training  Suggestions 

Number  of 
Specific 

Education/Training 

Suggestions 

2.4.4  Collective  analysis 

Mapping  the  five  main  complexity  factors  to  the  scenarios  showed  that  the  complexity  factors 
appear  to  be  relevant  to  the  experiences  of  the  SMEs.  The  collective  analyses  (an  overlap  and  a  gap 
analysis)  was  performed  to  integrate  the  previous  analyses  and  answer  several  remaining  important 
questions.  These  questions  were: 

1.  What  is  the  degree  of  overlap  between  the  challenges  identified  by  SMEs  and  the  five 
main  challenges  that  we  examined  from  the  complexity  literature? 

2.  What  are  the  gaps  in  the  complexity  literature  that  have  to  be  addressed  in  CF  education 
and  training  supporting  decision  making? 

2.4.4. 1  Examining  overlap  between  complexity  research  and  SME  experience 

Once  the  bottom-up  challenges  analysis  was  performed,  the  general  themes  were  examined  to 
determine  the  extent  to  which  they  contained  components  of  the  five  main  complexity  factors 
identified  from  the  complexity  literature  review.  This  was  done  to  determine  whether  the  actual 
experience  of  CF  personnel  can  be  effectively  described  using  complexity  ideas,  or  whether 
additional  ways  of  presenting  information  about  challenges  would  be  required  to  facilitate 
education  and  training. 
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To  determine  whether  the  general  theme  overlapped  with  the  complexity  category,  the  specific 
examples  related  to  each  theme  were  examined.  If  an  example  could  be  found  which  appeared  to  be 
related  to  one  of  the  five  main  complexity  factors,  it  was  provided  and  taken  as  evidence  for 
overlap  between  the  general  them  and  the  complexity  factor.  If  no  example  could  be  found  that  was 
obviously  related  to  one  of  the  five  complexity  factors,  then  that  general  theme  was  considered  to 
not  overlap  with  the  complexity  factor.  See  Table  5  for  the  template  used  for  this  analysis. 


Table  5:  General  themes,  definitions,  overlap,  and  gap  analysis  template 


General  Theme 

Theme  Description 

Subcomponents 

Overlap 

Gap  Analysis 

2. 4. 4. 2  Gap  analysis 

One  of  the  important  pieces  of  information  required  from  this  comparison  of  the  complexity 
literature  and  the  actual  experience  of  CF  personnel  is  a  determination  of  what  important 
challenges  to  the  decision  making  of  CF  personnel  are  not  well  captured  in  the  complexity 
literature.  These  gaps  need  to  be  identified  and  investigated  further  if  the  CF  education  and  training 
processes  are  to  be  adequately  supported. 

To  determine  if  there  were  gaps,  the  general  themes  and  related  examples  generated  from  the 
bottom-up  challenges  analysis  were  examined  and  anything  not  apparently  represented  in  the 
complexity  literature  was  identified  and  included  in  the  general  themes  analyses  (i.e.,  see  Table  5). 
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3.1  Review  of  complexity  literature 

To  understand  the  nature  of  complex  decision  making,  we  reviewed  a  number  of  articles.  These 
showed  that  there  are  several  important  components  of  decision  making  to  consider.  These  include 
what  characterizes  complex  decision  making  environments,  what  additional  factors  can  influence 
the  complexity  of  decision  making,  and  behaviours  which  can  be  adaptive  or  pathological  in 
complex  decision  making  situations. 

According  to  Brown  et  al.  (2009;  also  see  Brehmer  &  Allard,  1991),  there  are  four  generic 
characteristics  of  the  decision-making  task  in  complex  environments,  they  are: 

1)  There  are  a  series  of  decisions, 

2)  These  decisions  are  interdependent, 

3)  The  environment  changes  both  autonomously  and  as  a  function  of  the  decision  maker’s 
actions,  and 

4)  Timing  is  a  key  element,  where  decision  makers  have  little  control  over  exactly  when 
dynamic  decisions  must  be  made. 

If  these  factors  are  not  present,  then  it  is  unlikely  that  the  environment  involves  complex  decision 
making. 

A  further  review  of  the  literature  revealed  factors  that  are  typical  of  complex  decision  making 
contexts ,  and  which  can  add  to  the  complexity  of  decision  making.  These  include: 

1)  Connectivity  (a  large  number  of  diverse,  interacting  components), 

2)  Dynamics  (constant  change,  even  without  the  input  of  a  decision  maker), 

3)  Intransparency  (opaque  relationships  between  variables), 

4)  Information  overload  (i.e.,  a  large  amount  of  data), 

5)  Independent  agents, 

6)  Multiple  goals  which  might  conflict, 

7)  Underspecified  goals,  and 

8)  Challenges  with  self-reflection. 

Collectively,  these  main  12  factors  related  to  complexity  were  used  to  create  the  skeleton  of  a  Mind 
Map. 

Grisogono  (2010)  lists  behaviours  which  are  supposed  to  be  related  to  positive  outcomes  (“good 
actor  behaviours”)  or  related  to  negative  outcomes  (“poor  actor  behaviours”)  in  complex  decision 
making  environments.  These  behaviours  were  examined  and,  to  eliminate  redundancy,  similar 
behaviours  were  combined  (e.g.,  “extremely  interested  in  information  contradicting  own  view  of 
the  world”  and  “challenges  own  concepts  and  beliefs,  entertains  alternatives”  were  combined  into 
“challenge  own  concepts  and  beliefs,  entertains  alternatives”.  These  good  and  bad  decision  making 
behaviours  were  added  to  the  Mind  Map  and  associated  with  the  related  complexity  factor. 


Human  systems "  Incorporated 


Page  17 


HUM  an: systems 


Originally  it  was  hoped  that  we  could  gather  information  about  good  actor  and  poor  actor 
behaviours  related  to  complex  decision  making,  and  determine  which  of  these  should  be  taught  so 
that  CF  personnel  would  be  better  able  to  handle  complex  decision  making  situations. 
Unfortunately,  many  of  the  good  and  poor  actor  behaviours  are  very  difficult  to  empirically  assess. 
For  example,  Grisogono  (2010)  lists  “able  to  judge  how  much  planning  is  enough”  as  a  good  actor 
behaviour,  and  “over-planning”  (i.e.,  too  much  planning)  and  “bang-bang-decisions”  (i.e.,  too  little 
planning)  as  poor  actor  behaviours.  Unfortunately,  there  is  no  guidance  as  to  how  to  assess  how 
much  planning  is  actually  “enough”.  By  its  nature,  complex  decision  making  is  difficult  and 
unpredictable,  and  even  the  best  decision  makers  will  not  always  have  a  successful  outcome,  and  a 
successful  outcome  could  be  the  result  of  chance  even  with  poor  decision  making.  Therefore, 
whether  behaviours  are  good  or  poor  cannot  reasonably  be  assessed  by  examining  outcomes  only. 

Note  that,  due  to  resource  limitations,  the  literature  review  performed  for  this  project  was 
extremely  limited.  There  are  two  aspects  of  the  complex  decision  making  literature  that  could  not 
adequately  be  investigated  here,  but  are  worthy  of  mention.  The  first  is  the  team  component  of  the 
complex  decision  making  literature  (e.g.,  Artman,  1999;  Clancy,  Elliott,  Ley,  Omodei,  Wearing, 
McLennan,  &  Thorsteinsson,  2003;  and  McLennan,  Omodei,  Flolgate,  and  Wearing,  2003). 
Although  team  decision  making  was  beyond  the  scope  of  the  current  work,  it  is  likely  highly  that 
this  research  will  be  relevant  for  understanding  some  of  the  challenges  facing  decision  makers  in 
the  CF.  The  second  aspect  of  complex  decision  making  research  that  could  not  be  reviewed  here  is 
the  role  of  emotional  and  motivational  factors  in  complex  decision  making.  This  has  only  recently 
been  examined  by  researchers,  specifically  Dorner  who  is  in  the  process  of  creating  a  framework 
integrating  personality  and  motivational  elements  with  good/poor  actor  behaviours.  Unfortunately, 
as  of  the  writing  of  this  report  that  literature  is  not  readily  available  in  English,  and  so  is  not  cited 
or  reviewed  here.  Hopefully  future  work  can  take  advantage  of  these  two  areas  of  research  into 
complex  decision  making  and  integrate  those  findings  with  the  findings  reported  here. 

The  complexity  terms  used  in  the  Mind  Map  were  double  checked  with  the  terms  used  in  Funke 
(2001),  and  where  possible  the  terms  used  in  that  article  were  used  in  the  mind  map  and  noted.  In 
addition,  a  preliminary  review  of  operational  terminology  was  performed  (largely  from  the 
Australian  Department  of  Defence  (2009)  campaign  manual)  and  some  terms  were  mapped  to  the 
related  complexity  factors  to  show  that  these  factors  likely  are  relevant  to  the  military  domain. 

This  Mind  Map  was  then  used  during  the  Workshop  as  a  basis  for  discussion.  See  Annex  A  for  the 
Mind  Map.  One  thing  to  note  is  that,  during  this  review,  we  found  that  human  limitations  are 
critical  for  understanding  what  makes  something  complex,  as  it  is  characteristics  and  limitations  of 
the  decision  maker  that  makes  things  difficult.  However,  it  was  not  possible  to  cleanly  map 
challenges  onto  human  limitations;  for  example,  working  memory  limitations  played  a  role  in  most 
challenges.  Thus,  human  limitations  are  not  explicitly  a  component  of  the  Mind  Map. 

3.2  Workshop 

During  the  workshop,  the  goals  were  to  narrow  the  scope  of  the  investigation  to  the  components  of 
complexity  that  the  SA  thought  were  most  important  (due  to  time  and  resource  considerations  it 
was  thought  that  not  all  factors  could  be  given  equal  weight),  and  to  determine  which  CF 
environments  and  decision  making  contexts  were  of  major  interest  so  that  SMEs  could  be  recruited 
for  the  knowledge  elicitation  sessions. 
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3.2.1  Prioritization  and  definition  of  complexity  factors 

The  Mind  Map  was  reviewed  and  the  participants  suggested  a  few  minor  changes.  The  revised 
Mind  Map  is  presented  in  Annex  A. 

After  discussion,  the  workshop  participants  agreed  that  this  project  should  focus  on  five  main 
factors  related  to  complexity.  These  five  factors  were: 

1)  Connectivity 

2)  Dynamics 

3)  Multiple  conflicting  goals 

4)  Under-specified  goals 

5)  Independent  agents 

These  five  factors  were  chosen  as  they  were  thought  to  be  most  characteristic  of  complex 
environments  (e.g.,  not  just  information  overload  that  can  occur  in  complicated  as  well  as  complex 
decisions)  and  also  what  CF  personnel  seem  to  talk  about  when  discussing  what  makes  their 
decisions  difficult  (e.g.,  dealing  with  civilians). 

Once  these  five  main  factors  were  chosen,  there  was  further  discussion  to  ensure  that  everyone  had 
the  same  understanding  of  these  factors,  and  definitions  were  created.  The  definitions  of  five  key 
complexity  factors  were: 

1)  Connectivity:  Things  in  the  environment  influence  one  another  in  complicated  and 
unpredictable  ways, 

2)  Dynamics:  The  system  has  aspects  that  unfold  over  time.  For  example,  the 
environment  changes  over  time  even  when  you  do  nothing;  the  rate  at  which  things 
change  may  be  variable;  there  may  be  delays  between  actions  and  effects, 

3)  Multiple  conflicting  goals:  Having  to  achieve  multiple  objectives  which  may  not  be  all 
achievable  at  the  same  time, 

4)  Underspecified  goals:  Goals  may  be  difficult  to  achieve  because  they  are  too  vague, 

5)  Independent  agents:  There  are  independent  entities  in  the  environment  that  influence  it 
(they  may  have  different  goals  than  the  decision  maker). 

3.2.2  Potential  scenarios  and  SMEs 

Once  the  five  complexity  factors  had  been  chosen,  potential  CF  operation  types  were  discussed  to 
determine  overall  areas  that  should  be  considered  in  scope.  Domestic  operations  and  expeditionary 
operations  were  both  considered  to  be  within  scope.  It  was  determined  that,  when  examining 
expeditionary  operations,  we  should  try  not  to  focus  only  on  Afghanistan,  if  possible.  Land  force 
operations,  rather  than  air  force  or  naval  operations,  were  to  be  the  focus  of  the  project. 

After  discussion,  four  main  CF  contexts  were  considered  to  be  within  scope:  domestic  operations, 
domestic  day-to-day  functions,  expeditionary  offensive  operations,  and  expeditionary  humanitarian 
operations.  Several  examples  that  were  likely  to  be  complex  and  fit  each  of  these  contexts  were 
noted  by  workshop  participants  (see  Table  6). 
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Table  6:  Potential  CF  contexts  for  knowledge  elicitation 


Scenario  Categories 

Potential  Scenarios 

Domestic,  Day-to-day 

Personnel 

Capital  Acquisition 

Army  of  Tomorrow  -  2021 

Increasing  size  of  CF 

Canadian  Forces  Leadership  Institute 

Directorate  of  Land  Concepts  and  Designs 

Directorate  Doctrine  &  Training 

Peace  Support  Training  Centre 

Intelligence 

Canadian  Land  Force  Command  and  Staff  College  (CLFCSC)  -  Centre  of 
Excellence  (CoE) 

Domestic  Operations 

Olympics 

Arctic  Sovereignty 

Overseas  -  Humanitarian 

Haiti 

Disaster  Assistance  Response  Team 

Civil  Military  Cooperation  (CIMIC) 

Overseas  -  Offensive 

Leaving  Afghanistan 

Intel 

“Attacking  the  network”  (counter-improvised  explosive  device) 

CIMIC 

Reintegration 

Combat  Logistics 

Force  Generation 

Pre-deployment  Task  Force  Training 

Recruitment  of  SMEs  was  done  with  the  goal  of  obtaining  interviewees  who  had  experience  with 
specific  contexts  that  involve  complex  decision  making. 

3.3  Knowledge  elicitation 

The  knowledge  elicitations  sessions  were  intended  to  obtain  information  about  the  actual  decision 
making  experienced  by  CF  personnel,  to  allow  the  creation  of  descriptions  of  those  experiences  as 
well  as  to  facilitate  a  comparison  between  the  five  selected  complex  decision  making  factors  and 
those  experiences. 
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The  knowledge  elicitation  sessions  involved  interviewing  SMEs  who  had  experience  in  the 
decision  making  contexts  which  were  identified  in  the  workshops  as  likely  representing  complex 
decision  making  environments.  The  interview  technique  used  was  semi-structured  interviews  based 
on  the  CDM  technique.  See  the  Method  Section  (  Section  2.3)  for  an  in-depth  overview  of  the 
interview  structure  used. 

SMEs  were  recruited  via  workshop  members  and  their  networks  of  contacts.  There  were  a  total  of 
12  SMEs  interviewed;  of  these,  two  interviews  were  not  analysed  as  the  SME  had  difficulty 
providing  examples  of  specific  situations  which  involved  the  complex  decision  making  factors  we 
were  concentrating  on  (either  because  they  could  not  remember  examples  or  because  they  were 
reluctant  to  discuss  examples  that  could  be  sensitive).  Of  the  final  list  of  10  SMEs,  the  roles 
discussed  are  listed  below  in  Table  7.  Note  that  no  SMEs  with  humanitarian  experience  were  on  the 
final  list  of  SMEs  recruited  for  this  project. 


Table  7:  SME  roles  described  in  knowledge  elicitation  sessions 


Scenario  Category 

Context  (Role) 

Domestic,  Day-to-Day 

CoE  Training  Development  Supervisor 

CoE  Training  Developer 

HF  Engineer,  Capital  Acquisition 

Professional  Military  Education  (PME)  Revitalization  Supervisor 

PME  Revitalization  Staff  Member 

Psychological  Operations  (PSYOPS)  Training  Program  Developer 

Domestic  Ops 

Military  Liaison  and  Advisor,  International  Event 

Expeditionary  -  Offensive 

Liaison  Officer,  Afghanistan 

National  Support  Element  (NSE)  Officer,  Afghanistan 

Chief  of  Staff,  Strategic  Advisory  Team,  Afghanistan 

3.4  Analysis 

3.4.1  Scenarios 

The  scenarios  were  created  from  the  data  collected  during  the  SME  interviews.  The  scenarios 
contain  a  general  description  of  the  role  of  the  SME  (referred  to  as  the  DM)  and  a  list  of  the  factors 
which  were  mentioned  by  the  DM  as  playing  a  large  role  in  his  or  her  decision  making.  Factors  that 
unfolded  over  time  were  pointed  out  as  dynamic  events  are  fundamental  to  complex  decision 
making. 

Note  that  the  scenarios  were  reviewed  several  times  to  ensure  that  details  provided  were 
appropriate,  both  in  terms  of  accuracy  and  that  they  did  not  compromise  sensitive  information.  The 
scenarios  were  reviewed  by  the  SME  who  was  the  interviewee,  one  SME  who  was  sensitive  to  the 
issues  reviewed  all  scenarios,  and  the  scenarios  were  also  reviewed  by  the  Scientific  Authority. 
Because  some  of  the  details  provided  were  considered  potentially  sensitive,  some  details  were 
omitted  from  the  scenarios  themselves  (and  the  report  as  a  whole),  but  were  still  counted  as  factors 
that  were  considered  during  decision  making.  For  example,  when  a  large  number  of  factors  were 
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listed  as  being  considered  in  decision  making,  but  those  factors  would  reveal  too  much  about  the 
identity  of  the  DM  or  the  situation,  then  the  number  of  factors  were  listed  but  the  factors 
themselves  were  not. 

Below  is  a  segment  of  the  Liaison  Officer,  Afghanistan  scenario  description,  to  illustrate  what  is 
provided  in  the  scenarios  in  terms  of  the  general  description  of  the  DM’s  role  and  examples  of  the 
factors  provided. 

3.4. 1.1  Example  scenario:  Liaison  Officer,  Afghanistan 

The  DM  was  assigned  to  act  as  a  liaison  officer  in  the  operations  coordination  centre  in  Kandahar, 
Afghanistan.  It  was  the  responsibility  of  the  DM  to  act  as  a  liaison  between  the  Canadian  Forces 
(CF)  and  the  Afghanistan  National  Army  (ANA),  Afghanistan  National  Police  (ANP),  Kandahar 
Prison,  Border  Security,  and  the  National  Directorate  of  Security  (NDS;  the  Afghanistan  Secret 
Service).  At  the  beginning  of  the  DM’s  deployment,  there  was  a  massive  escape  from  the  Kandahar 
prison.  This  created  an  atmosphere  of  uncertainty  about  the  level  of  security  present  in  Kandahar, 
and  part  of  the  DM’s  role  was  to  facilitate  an  increase  in  security.  In  particular,  the  DM  was 
required  to  liaise  with  the  relevant  parties  to  increase  security  in  preparation  for  voter  registration 
which  was  to  occur  approximately  8  months  later.  The  DM  was  to  create  a  “Kandahar  city  security 
network”;  he  had  to  convince  the  relevant  stakeholders  that  it  was  necessary,  persuade  them  to  take 
part,  and  oversee  the  process. 

Factors  which  influenced  the  ability  of  the  DM  to  manage  these  processes  included: 

•  The  prison  break  was  believed  to  be  facilitated  by  assistance  from  inside  the  prison.  Many 
of  the  senior  leadership  at  the  prison  were  either  arrested  or  fired.  This  caused  a  massive 
change  in  personnel  and  a  huge  loss  of  confidence  in  the  prison  system  and  personnel,  as 
well  as  a  re-evaluation  of  many  assumptions  held  by  CF  personnel  about  security  in 
Kandahar; 

•  There  was  a  quick  turnover  in  Afghan  personnel;  for  example,  within  9  months  there  were 
3  governors  of  Kandahar. 

Factors  that  unfolded  over  time: 

•  The  prison  break  led  to  a  process  of  information  gathering.  This  process  gradually  revealed 
that  there  was  no  Common  Operating  Picture  (COP)  among  Afghan  security  agencies;  that 
the  ANP  could  not  effectively  get  information  about  threats;  that  the  ANP  had  difficulty 
responding  to  threats;  that  the  ANP  could  not  effectively  ask  for  assistance  from  other 
security  organizations;  and  that  Afghan  security  organizations  typically  work  at  a  tactical 
level  and  are  not  used  to  working  at  an  operational  or  strategic  level.  These  pieces  of 
information  changed  how  the  DM  saw  his  goals  and  how  they  could  be  accomplished; 

•  The  DM  attempted  to  get  the  relevant  stakeholders  to  install,  maintain,  train  on,  and  use 
communication  equipment  provided  by  the  U.S.  He  found  this  very  difficult  for  many 
reasons  (e.g.,  cultural  differences),  and  felt  that  any  intervention  he  tried  did  not  result  in 
change  (i.e.,  there  seemed  to  be  a  “set  point”  in  the  environment  that  was  highly  resistant  to 
change).  This  became  more  and  more  frustrating  as  time  went  on  and  the  DM  grew 
increasingly  hopeless  that  the  C2  infrastructure  would  be  adequate  to  maintain  security. 

The  full  scenario  descriptions  can  be  found  in  Annex  B:  Operational  Scenarios  that  Contain 
Complexity. 
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3.4.2  Mapping  of  five  complexity  factors  to  scenarios 

One  of  the  main  goals  of  this  project  was  to  determine  whether  factors  from  the  complexity 
literature  can  be  mapped  to  experiences  of  members  of  the  CF.  To  facilitate  this,  an  explicit 
mapping  from  the  scenarios  to  the  five  key  complexity  factors  was  performed.  Examples  from  the 
scenarios  were  provided  for  each  of  the  factors  (for  more  about  the  development  of  this  analysis 
process,  see  details  in  Section  2.4.2). 

Recall  that,  due  to  the  relative  rating  scheme  used,  a  rating  of  “HIGH”  means  that  this  factor 
occurred  in  this  scenario  to  a  relatively  large  extent  (i.e.,  relative  to  the  other  scenarios),  a  rating  of 
“MEDIUM”  means  that  this  factor  occurred  in  this  scenario  to  an  intermediate  extent,  and  a  rating 
of  “LOW”  means  that  this  factor  occurred  in  this  scenario  to  a  relatively  small  extent;  however,  any 
absolute  conclusions  must  be  interpreted  cautiously  as  ratings  are  only  relative  to  the  10  scenarios 
discussed  here.  These  general  criteria  and  ratings  can  be  found  in  Table  8,  an  example  mapping  of 
scenario  components  onto  the  five  complexity  factors  (for  the  Liaison  Officer,  Afghanistan 
scenario)  can  be  found  in  Table  9.  All  of  the  scenario  mappings  can  be  found  in  Annex  C. 


Table  8:  Complexity  rating  criteria 


Complexity  Factor  and 
Definition 

Criteria 

Rating  (High,  Medium,  Low)  and  Justification 

Connectivity: 

Things  in  the 
environment  influence 
one  another  in 
complicated  and 
unpredictable  ways 

How  many  interrelated 
factors  have  to  be 
considered  when 
making  decisions? 

How  many  2nd  and  3rd 
order  effects? 

High 

At  least  1  example  with  10  or  more  factors  that  have  to  be 
considered  for  a  single  decision 

At  least  1  example  with  10  or  more  2nd  and  3rd  order  effects  (or 
potential  effects)  that  could  result  from  a  single  decision 

Medium 

At  least  1  example  with  between  5  and  9  factors  that  have  to  be 
considered  for  a  single  decision 

At  least  1  example  with  between  5  and  9  2nd  and  3rd  order  effects 
(or  potential  effects)  that  could  result  from  a  single  decision 

OR 

Criteria  for  high  and  low  categories  are  not  met  (e.g.,  any  scenario 
that  meets  one  but  not  both  high  criteria) 

Low 

No  example  with  more  than  4  factors  that  have  to  be  considered 
fora  single  decision 

No  examples  with  more  than  4  2nd  and  3rd  order  effects  (or 
potential  effects)  that  could  result  from  a  single  decision 

Dynamics: 

The  system  has  aspects 
that  unfold  over  time.  For 
example,  the 
environment  changes 
over  time  even  when  you 
do  nothing;  the  rate  at 
which  things  change  may 
be  variable;  there  may  be 

Were  there  examples 
of  situations  that 
unfolded  over  time  that 
profoundly  affected 
decision  making? 

How  many 

subsystems  impacted 
the  decision-making 

High 

Seven  or  more  specific  examples  of  situations  that  unfolded  over 
time  that  profoundly  affected  decision  making 

At  least  10  subsystems  which  impacted  decision  making  and  can 
be  shown  to  have  different  dynamics  (timelines,  processes, 
cultures,  rates  of  change) 

Medium 

Five  or  6  specific  examples  of  situations  that  unfolded  over  time 
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Complexity  Factor  and 
Definition 

Criteria 

Rating  (High,  Medium,  Low)  and  Justification 

delays  between  actions 
and  effects. 

It  is  important  to  note 
hierarchical  aspects  if 
present  (i.e.,  subsystems 
that  have  their  own 
dynamics  which  are  part 
of  the  DM  context). 

context? 

Were  there 
subsystems  that  had 
hierarchical  aspects 
(e.g.,  their  own 
dynamics)? 

that  profoundly  affected  decision  making 

Five  to  9  subsystems  which  impacted  decision  making  that  can  be 
shown  to  have  different  dynamics 

OR 

Criteria  for  high  and  low  categories  are  not  met  (e.g.,  any  scenario 
that  meets  one  but  not  both  high  criteria) 

Low 

Four  or  fewer  examples  of  situations  that  unfolded  over  time  that 
profoundly  affected  decision  making 

Four  or  fewer  subsystems  which  impacted  decision  making  that 
can  be  shown  to  have  different  dynamics 

Multiple  conflicting  goals: 

Having  to  achieve 
multiple  objectives  which 
may  not  be  all  achievable 
at  the  same  time 

Are  there  many  goals? 

Are  there  problems 
with  goal  prioritization? 

Do  these  goals 
necessarily  conflict? 

Are  there  situations 
where  the  goals 
conflict  even  if  they 
wouldn’t  necessarily 
have  to? 

High 

At  least  6  distinct  goals 

At  least  3  cases  where  there  was  goal  conflict  or  goal  prioritization 
issues  which  influenced  decision  making 

Medium 

Four  or  5  distinct  goals 

Two  cases  where  there  was  goal  conflict  or  goal  prioritization 
issues  which  influenced  decision  making 

OR 

Criteria  for  high  and  low  categories  are  not  met  (e.g.,  any  scenario 
that  meets  one  but  not  both  high  criteria) 

Low 

Three  or  fewer  distinct  goals 

One  or  no  cases  where  goal  conflict  or  goal  prioritization  issues 
influenced  decision  making 

Under-specified  goals: 
Goals  may  be  difficult  to 
achieve  because  they  are 
too  vague 

Are  there  goals  that 
are  vague? 

To  what  extent  did  the 
vagueness  of  goals 
complicate  decision 
making? 

High 

Three  or  more  goals  that  were  vague,  where  the  vagueness 
impacted  the  ability  of  the  DM  to  make  decisions 

Medium 

One  or  2  goals  that  were  vague,  where  the  vagueness  impacted 
the  ability  of  the  DM  to  make  decisions 

Low 

Goals  were  generally  clear  (at  least  at  a  high  level),  and 
vagueness  did  not  appear  to  impact  the  ability  of  the  DM  to  make 
decisions 

Independent  agents: 

There  are  independent 
entities  in  the 
environment  who 
influence  it  (they  may 

Were  there 
independent  agents 
who  impact  decision 
making? 

Did  independent 

High 

Eight  or  more  groups  of  independent  agents  who  impacted 
decision  making 

Six  or  more  examples  of  independent  agents  with  goals  that 
conflicted  or  could  interfere  with  the  DM’s  goals 
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Complexity  Factor  and 
Definition 

Criteria 

Rating  (High,  Medium,  Low)  and  Justification 

have  different  goals  than 
the  decision  maker) 

agents  have  goals  that 
conflicted  with  the 

DM’s  goals? 

Did  independent 
agents  have  goals  that 
conflicted  with  one 
another  in  such  a  way 
that  the  DM’s  decision 
making  was  affected? 

Medium 

Five  to  7  groups  of  independent  agents  who  impacted  decision 
making 

Three  to  5  examples  of  independent  agents  with  goals  that 
conflicted  with  or  could  interfere  with  the  DM’s  goals 

OR 

Criteria  for  high  and  low  categories  are  not  met  (e.g.,  any  scenario 
that  meets  one  but  not  both  high  criteria) 

Low 

Less  than  5  groups  of  independent  agents  who  impacted  decision 
making 

Two  or  fewer  examples  of  independent  agents  with  goals  that 
conflicted  with  or  could  interfere  with  the  DM’s  goals 

Table  9:  Example  scenario  mapping  onto  the  five  complexity  factors  (Liaison  Officer,  Afghanistan) 


Concept  and 
Definition 

Concept  Examples 

Concept  Importance  for 
Scenario  (High,  Medium, 
Low)  and  Justification 

Connectivity: 

Things  in  the 
environment 
influence  one 
another  in 
complicated  and 
unpredictable 
ways 

Examples  of  decisions  which  involved  interrelated  factors  included: 

The  prison  break  led  to  a  process  of  information  gathering.  This 
process  gradually  revealed  that  there  was  no  Common  Operating 
Picture  (COP)  among  Afghan  security  agencies;  that  the  ANP  could 
not  effectively  get  information  about  threats;  that  the  ANP  had 
difficulty  responding  to  threats;  that  the  ANP  could  not  effectively  ask 
for  assistance  from  other  security  organizations;  and  that  Afghan 
security  organizations  typically  work  at  a  tactical  level  and  are  not 
used  to  working  at  an  operational  or  strategic  level.  These  pieces  of 
information  changed  how  the  DM  saw  his  goals  and  how  they  could 
be  accomplished; 

Cultural  differences  between  Afghanistan  and  Canada  are  profound; 
they  created  a  lack  of  trust  and  difficulty  with  coordination,  affecting 
many  of  the  DM’s  decisions.  Factors  included: 

•  Extremely  high  emphasis  on  interpersonal  relationships  in 
Afghanistan  (e.g.,  the  importance  of  personal  relationships 
to  Afghans  is  paramount).  This  has  profound  implications; 
for  example,  authority  based  on  position  alone  is  not  “real” 
authority  to  the  Afghans  (obedience  is  based  on  personal 
relationships); 

•  Differences  in  the  legal  system  and  widespread  corruption 
(e.g.,  people  responsible  for  the  prison  break  probably 
were  not  punished  as  they  could  pay  to  be  released,  which 
destroyed  trust); 

•  A  difference  in  the  idea  of  what  it  is  to  have  a  job  and  the 
attendant  responsibility  (e.g.,  a  police  officer  threw  away 

High 

Many  factors  that  are 
interrelated  and  that  have 
to  be  considered  when 
making  decisions  (e.g., 
there  were  15  important 
cultural  effects  and 
implications  which 
influenced  the  DM’s 
decisions) 

Examples  of  2nd  and  3rd 
order  effects  (e.g.,  the  DM 
having  to  work  in  a  different 
culture  involved  at  least  15 
additional  decisions  or 
effects) 
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Concept  and 
Definition 

Concept  Examples 

Concept  Importance  for 
Scenario  (High,  Medium, 
Low)  and  Justification 

his  phone  because  he  was  getting  too  many  91 1  calls, 
even  though  that  was  the  only  phone  the  calls  were 
coming  to); 

•  The  Afghans  appeared  to  have  the  perspective  that  they 
had  little  control  over  events  (e.g.,  “what  Allah  wills  will 
happen”),  whereas  Canadians  typically  feel  they  do  have 
control  over  events.  This  had  implications;  for  example,  the 
Afghans  were  not  used  to  the  concept  of  practicing  for  a 
possible  event  and  were  resistant  to  such  training 

•  Different  social  standards  (e.g.,  the  DM  never  met  the  wife 
of  any  Afghan  he  worked  with).  The  DM  knew  that  social 
relationships  were  important,  but  was  not  able  to  actually 
understand  what  the  relationships  were  as  he  was  not 
privy  to  how  families  were  interrelated  through  marriage; 

•  Literacy  is  so  low  in  Afghanistan  that  usually  no  written 
records  are  kept,  which  means  data  is  not  available  that 
would  help  assess  resource  and  training  requirements; 

•  Lack  of  appreciation  for  resources  (e.g.,  communications 
equipment  was  given  to  the  relevant  stakeholders;  one 
faction  of  the  ANA  changed  locations  but  left  the 
equipment  in  the  old  location  unattended  without  notifying 
the  DM); 

•  Age  is  more  of  a  factor  in  creating  a  sense  of  authority  in 
Afghanistan.  The  DM  felt  that  he  would  have  received 
more  respect  if  he  had  been  older; 

Dynamics: 

The  system  has 
aspects  that 
unfold  over  time. 

For  example,  the 
environment 
changes  over 
time  even  when 
you  do  nothing; 
the  rate  at  which 
things  change 
may  be  variable; 
there  may  be 
delays  between 
actions  and 
effects. 

It  is  important  to 
note  hierarchical 
aspects  if  present 
(i.e.,  subsystems 
that  have  their 
own  dynamics 

Situations  that  unfolded  over  time  that  profoundly  affected  decision 
making  included: 

There  was  a  91 1  system  that  was  implemented;  this  was  supposed 
to  be  run  by  the  ANP  but  the  chief  of  police  discarded  the  phone  that 
was  used  for  the  91 1  calls  as  he  found  it  inconvenient.  The  DM 
volunteered  to  take  over  the  91 1  function;  this  became  a  tool  to  get 
the  ANP  involved  as  the  operations  room  would  receive  the  calls  but 
the  ANP  would  be  contacted  to  answer  them.  This  created  additional 
(an  unpredictable)  windows  of  opportunity  for  collaboration  with  the 
ANP,  for  obtaining  intelligence,  and  for  facilitating  relationships  with 
civilians; 

The  prison  break  led  to  a  process  of  information  gathering.  This 
process  gradually  revealed  that  there  was  no  Common  Operating 
Picture  (COP)  among  Afghan  security  agencies;  that  the  ANP  could 
not  effectively  get  information  about  threats;  that  the  ANP  had 
difficulty  responding  to  threats;  that  the  ANP  could  not  effectively  ask 
for  assistance  from  other  security  organizations;  and  that  Afghan 
security  organizations  typically  work  at  a  tactical  level  and  are  not 
used  to  working  at  an  operational  or  strategic  level.  These  pieces  of 
information  changed  how  the  DM  saw  his  goals  and  how  they  could 
be  accomplished; 

Medium 

Three  specific  examples  of 
situations  that  unfolded 
over  time  that  profoundly 
affected  decision  making 

Six  subsystems  with 
somewhat  different 
dynamics  which  impacted 
decision  making 
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Concept  and 
Definition 

Concept  Examples 

Concept  Importance  for 
Scenario  (High,  Medium, 
Low)  and  Justification 

which  are  part  of 
the  DM  context). 

The  DM  attempted  to  get  the  relevant  stakeholders  to  install, 
maintain,  train  on,  and  use  communication  equipment  provided  by 
the  U.S.  He  found  this  very  difficult  for  many  reasons  (e.g.,  cultural 
differences),  and  felt  that  any  intervention  he  tried  did  not  result  in 
change  (i.e.,  there  seemed  to  be  a  “set  point”  in  the  environment  that 
was  highly  resistant  to  change).  This  became  more  and  more 
frustrating  as  time  went  on  and  the  DM  grew  increasingly  hopeless 
that  the  C2  infrastructure  would  be  adequate  to  maintain  security. 

There  were  subsystems  which  had  their  own  dynamics  in  this 
context.  These  included: 

The  ANA; 

The  ANP 

Kandahar  prison  personnel; 

The  CF; 

Civilians; 

The  operations  centre  in  which  the  DM  was  working. 

Canadian  and  other  foreign  military  organizations  appeared  to  have 
a  relatively  fast  rate  of  change  compared  to  the  Afghan 
organizations  (e.g.,  longer  time  to  train,  longer  time  to  change 
procedures),  with  an  exception  being  in  the  rate  of  personnel 
change,  with  higher  turnover  in  the  Afghan  organizations. 

Multiple 

conflicting  goals: 
Having  to  achieve 
multiple 

objectives  which 
may  not  be  all 
achievable  at  the 
same  time 

The  goals  of  the  DM  included: 

Creating  an  effective  Kandahar  city  security  network 

Building  and  maintaining  relationships  with  members  of  the  ANA, 

ANP,  Kandahar  prison  personnel,  Border  Security,  and  the  NDS. 

There  were  some  cases  in  which  it  was  difficult  to  achieve  all  of 
these  goals.  One  example  was  that  the  ANP  and  ANA  were 
adversaries  that  the  DM  was  trying  to  get  to  work  together,  and  it 
was  difficult  to  meet  both  of  their  needs  and  expectations.  Another 
example  is  that  the  DM  had  to  balance  the  feedback  accuracy  that 
he  provided  to  trainees  with  allowing  them  to  save  face  and  remain 
invested  (i.e.,  too  much  negative  feedback  would  likely  have  resulted 
in  stakeholders  withdrawing  from  the  training). 

Medium 

There  were  a  number  of 
important  goals  that 
conflicted  in  different  ways 
in  different  situations  (6 
distinct  goals) 

Two  examples  of  goal 
conflict 

Primary  difficulty  due  to 
problems  with  getting 
stakeholder  buy-in  and 
participation  rather  than 
goals  conflicting 

Under-specified 

goals: 

Goals  may  be 
difficult  to  achieve 
because  they  are 
too  vague 

Examples  of  underspecified  goals  included: 

The  goal  of  creating  an  effective  Kandahar  city  security  network  was 
superficially  clear  but  what  was  actually  required  and  the  poor  state 
of  the  current  system  was  only  revealed  over  time. 

Medium 

One  example  of  a  vague 
goal  that  impacted  the 
ability  of  the  DM  to  make 
decisions 

Goals  at  a  high  level  were 
fairly  clear,  the  main 
challenge  was  to  determine 
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Concept  Examples 

Concept  Importance  for 
Scenario  (High,  Medium, 
Low)  and  Justification 

current  security  situation 
and  needs 

Independent 

agents: 

There  are 
independent 
entities  in  the 
environment  who 
influence  it  (they 
may  have 
different  goals 
than  the  decision 
maker) 

Independent  agents  who  influenced  decision  making  included: 

The  CF; 

The  ANA  (didn’t  like  working  with  the  ANP  and  vice  versa); 

The  ANP; 

Kandahar  prison  personnel  (not  trusted;  most  were  new  after  the 
prison  break); 

Border  security; 

NDS; 

IEC  (Independent  Electoral  Commission)  -  required  protection; 

Mentors. 

Large  problems  due  to  interpersonal  conflicts  between  different 
stakeholders  (e.g.,  ANA  and  ANP  didn’t  want  to  work  together;  no 
one  wanted  to  work  with  the  Kandahar  prison  personnel) 

Large  problems  related  to  getting  stakeholder  buy-in  and  adequate 
participation;  this  partially  a  cultural  problem  (e.g.,  all  Afghani  groups 
didn’t  want  to  cooperate  with  the  DM’s  training  plans) 

High 

Eight  independent  agents 
who  impacted  decision 
making 

Seven  examples  of 
independent  agents  with 
goals  that  conflicted  or 
could  interfere  with  the 

DM’s  goals 

Further  analyses  were  conducted  based  on  these  mappings,  including  the  Challenges  Analyses 
discussed  in  the  next  section. 

3.4.3  Challenges  Analyses 

Both  top-down  and  bottom-up  analyses  were  applied  to  the  data  collected  in  order  to  identify  and 
quantify  decision  making  challenges. 

3.4.3. 1  Top-down  challenges  analysis 

The  top-down  challenges  analysis  quantifies  the  mappings  done  (in  Section  3.4.2  above).  To 
determine  whether  different  scenario  types  appeared  to  differ  on  their  overall  level  of  complexity, 
the  complexity  ratings  were  scored  (“F1IGF1”  =  2,  “MEDIUM”  =  1 ,  and  “LOW”  =  0),  the  scores 
were  totalled  to  get  an  overall  complexity  score  (the  higher  the  score,  the  more  complex  the 
scenario),  and  then  the  scenarios  were  ordered  based  on  their  overall  complexity  and  patterns  were 
noted  (e.g.,  what  types  of  scenarios  appeared  to  be  highest  in  complexity).  Overall  complexity 
rating  scores  and  the  ordered  list  of  scenarios  by  complexity  are  presented  in  Table  10. 


Table  10:  Scenario  complexity  rating  scores 


Scenario 

Rating  Type 

Rating 

Complexity 
Total  /10 

Scenario  Type 

Military  Liaison 
and  Advisor, 
International 

Connectivity 

HIGH 

10 

Domestic 

Operations 

Dynamics 

HIGH 

Multiple  conflicting  qoals 

HIGH 

Under-specified  goals 

HIGH 
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Scenario 

Rating  Type 

Rating 

Complexity 
Total  /10 

Scenario  Type 

Event 

Independent  aqents 

HIGH 

NSE  Officer, 
Afghanistan 

Connectivity 

HIGH 

9 

Expeditionary 

Operations 

Dynamics 

HIGH 

Multiple  conflictinq  qoals 

HIGH 

Under-specified  qoals 

MEDIUM 

Independent  aqents 

HIGH 

Liaison  Officer, 
Afghanistan 

Connectivity 

HIGH 

7 

Expeditionary 

Operations 

Dynamics 

MEDIUM 

Multiple  conflictinq  qoals 

MEDIUM 

Under-specified  qoals 

MEDIUM 

Independent  aqents 

HIGH 

CoE  Training 
Developer 

Connectivity 

MEDIUM 

6 

Domestic  Day-to- 
Day  (Training) 

Dynamics 

MEDIUM 

Multiple  conflictinq  qoals 

MEDIUM 

Under-specified  qoals 

HIGH 

Independent  aqents 

MEDIUM 

Chief  of  Staff, 
Strategic  Advisory 
Team, 

Afghanistan 

Connectivity 

MEDIUM 

5 

Expeditionary 

Operations 

Dynamics 

MEDIUM 

Multiple  conflictinq  qoals 

MEDIUM 

Under-specified  goals 

MEDIUM 

Independent  aqents 

MEDIUM 

PME 

Revitalization 

Supervisor 

Connectivity 

MEDIUM 

4 

Domestic  Day-to- 
Day  (Training) 

Dynamics 

MEDIUM 

Multiple  conflictinq  qoals 

MEDIUM 

Under-specified  qoals 

LOW 

Independent  aqents 

MEDIUM 

CoE  Training 
Development 
Supervisor 

Connectivity 

MEDIUM 

4 

Domestic  Day-to- 
Day  (Training) 

Dynamics 

MEDIUM 

Multiple  conflictinq  qoals 

MEDIUM 

Under-specified  qoals 

LOW 

Independent  aqents 

MEDIUM 

HF  Engineer, 

Capital  Acquisition 

Connectivity 

LOW 

2 

Domestic  Day-to- 
Day 

(Procurement) 

Dynamics 

MEDIUM 

Multiple  conflictinq  qoals 

MEDIUM 

Under-specified  qoals 

LOW 

Independent  aqents 

LOW 

PME 

Revitalization 

Staff  Member 

Connectivity 

LOW 

2 

Domestic  Day-to- 
Day  (Training) 

Dynamics 

MEDIUM 

Multiple  conflictinq  qoals 

LOW 

Under-specified  qoals 

LOW 

Independent  aqents 

MEDIUM 

PSYOPS  Training 
Program 

Developer 

Connectivity 

MEDIUM 

1 

Domestic  Day-to- 
Day  (Training) 

Dynamics 

LOW 

Multiple  conflicting  qoals 

LOW 

Under-specified  goals 

LOW 

Independent  aqents 

LOW 

As  can  be  seen  from  the  total  complexity  scores  presented  in  Table  10,  the  order  of  complexity 
appears  to  be  almost  completely  separated  by  general  scenario  type,  at  least  for  the  1 0  scenarios 
compared  here.  The  most  complex  scenario  was  related  to  domestic  operations,  the  next  most 
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complex  scenario  type  appeared  to  be  expeditionary  operations,  and  the  least  complex  scenario 
type  appeared  to  be  domestic  day-to-day  functions. 

In  addition  to  the  overall  complexity  ratings  of  the  scenarios,  the  scenarios  were  also  ordered  based 
on  their  scores  on  the  five  complexity  factors.  These  ratings  can  be  seen  in  Table  1 1  through  Table 
15  below. 

Table  11:  Scenario  connectivity  ratings 


Scenario 

Connectivity  Rating 

Scenario  Type 

Military  Liaison  and  Advisor, 

International  Event 

HIGH 

Domestic  Operations 

NSE  Officer,  Afghanistan 

HIGH 

Expeditionary  Operations 

Liaison  Officer,  Afghanistan 

HIGH 

Expeditionary  Operations 

CoE  Training  Developer 

MEDIUM 

Domestic  Day-to-Day  (Training) 

Chief  of  Staff,  Strategic  Advisory 

Team,  Afghanistan 

MEDIUM 

Expeditionary  Operations 

PME  Revitalization  Supervisor 

MEDIUM 

Domestic  Day-to-Day  (Training) 

CoE  Training  Development  Supervisor 

MEDIUM 

Domestic  Day-to-Day  (Training) 

PSYOPS  Training  Program  Developer 

MEDIUM 

Domestic  Day-to-Day  (Training) 

HF  Engineer,  Capital  Procurement 

LOW 

Domestic  Day-to-Day  (Equipment  Procurement) 

PME  Revitalization  Staff  Member 

LOW 

Domestic  Day-to-Day  (Training) 

Table  12:  Scenario  dynamics  ratings 


Scenario 

Dynamics  Rating 

Scenario  Type 

Military  Liaison  and  Advisor,  International 
Event 

HIGH 

Domestic  Operations 

NSE  Officer,  Afghanistan 

HIGH 

Expeditionary  Operations 

Liaison  Officer,  Afghanistan 

MEDIUM 

Expeditionary  Operations 

CoE  Training  Developer 

MEDIUM 

Domestic  Day-to-Day  (Training) 

Chief  of  Staff,  Strategic  Advisory  Team, 
Afghanistan 

MEDIUM 

Expeditionary  Operations 

PME  Revitalization  Supervisor 

MEDIUM 

Domestic  Day-to-Day  (Training) 

CoE  Training  Development  Supervisor 

MEDIUM 

Domestic  Day-to-Day  (Training) 

HF  Engineer,  Capital  Procurement 

MEDIUM 

Domestic  Day-to-Day  (Equipment  Procurement) 

PME  Revitalization  Staff  Member 

MEDIUM 

Domestic  Day-to-Day  (Training) 

PSYOPS  Training  Program  Developer 

LOW 

Domestic  Day-to-Day  (Training) 
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Table  13:  Scenario  multiple  conflicting  goals  ratings 


Scenario 

Multiple  Conflicting  Goals 
Rating 

Scenario  Type 

Military  Liaison  and  Advisor,  International 
Event 

HIGH 

Domestic  Operations 

NSE  Officer,  Afghanistan 

HIGH 

Expeditionary  Operations 

Liaison  Officer,  Afghanistan 

MEDIUM 

Expeditionary  Operations 

CoE  Training  Developer 

MEDIUM 

Domestic  Day-to-Day  (Training) 

Chief  of  Staff,  Strategic  Advisory  Team, 
Afghanistan 

MEDIUM 

Expeditionary  Operations 

PME  Revitalization  Supervisor 

MEDIUM 

Domestic  Day-to-Day  (Training) 

CoE  Training  Development  Supervisor 

MEDIUM 

Domestic  Day-to-Day  (Training) 

HF  Engineer,  Capital  Procurement 

MEDIUM 

Domestic  Day-to-Day  (Equipment 
Procurement) 

PME  Revitalization  Staff  Member 

LOW 

Domestic  Day-to-Day  (Training) 

PSYOPS  Training  Program  Developer 

LOW 

Domestic  Day-to-Day  (Training) 

Table  14:  Scenario  under-specified  goals  ratings 


Scenario 

Under-specified  Goals 
Rating 

Scenario  Type 

Military  Liaison  and  Advisor, 

International  Event 

HIGH 

Domestic  Operations 

CoE  Training  Developer 

HIGH 

Domestic  Day-to-Day  (Training) 

NSE  Officer,  Afghanistan 

MEDIUM 

Expeditionary  Operations 

Liaison  Officer,  Afghanistan 

MEDIUM 

Expeditionary  Operations 

Chief  of  Staff,  Strategic  Advisory  Team, 
Afghanistan 

MEDIUM 

Expeditionary  Operations 

PME  Revitalization  Supervisor 

LOW 

Domestic  Day-to-Day  (Training) 

CoE  Training  Development  Supervisor 

LOW 

Domestic  Day-to-Day  (Training) 

HF  Engineer,  Capital  Procurement 

LOW 

Domestic  Day-to-Day  (Equipment  Procurement) 

PME  Revitalization  Staff  Member 

LOW 

Domestic  Day-to-Day  (Training) 

PSYOPS  Training  Program  Developer 

LOW 

Domestic  Day-to-Day  (Training) 
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Table  15:  Scenario  independent  agents  ratings 


Scenario 

Independent  Agents 
Rating 

Scenario  Type 

Military  Liaison  and  Advisor,  International 
Event 

HIGH 

Domestic  Operations 

NSE  Officer,  Afghanistan 

HIGH 

Expeditionary  Operations 

Liaison  Officer,  Afghanistan 

HIGH 

Expeditionary  Operations 

CoE  Training  Developer 

MEDIUM 

Domestic  Day-to-Day  (Training) 

Chief  of  Staff,  Strategic  Advisory  Team, 
Afghanistan 

MEDIUM 

Expeditionary  Operations 

PME  Revitalization  Supervisor 

MEDIUM 

Domestic  Day-to-Day  (Training) 

CoE  Training  Development  Supervisor 

MEDIUM 

Domestic  Day-to-Day  (Training) 

PME  Revitalization  Staff  Member 

MEDIUM 

Domestic  Day-to-Day  (Training) 

HF  Engineer,  Capital  Procurement 

LOW 

Domestic  Day-to-Day  (Equipment 

Procurement) 

PSYOPS  Training  Program  Developer 

LOW 

Domestic  Day-to-Day  (Training) 

Although  this  challenges  analysis  uses  only  a  comparison  within  the  set  of  10  scenarios  developed 
in  this  project,  several  conclusions  can  be  made.  Most  SMEs  had  little  trouble  describing  decision 
making  contexts  in  which  they  had  experienced  the  five  complexity  factors,  and  generally  indicated 
that  these  situations  were  challenging.  Thus,  it  seems  reasonable  to  conclude  that  the  complexity 
challenges  noted  here  do  indeed  make  decision  making  more  difficult,  and  also  that  the  complexity 
challenges  are  frequently  experienced  by  CF  personnel.  Therefore,  addressing  these  types  of 
challenges  in  general  could  improve  CF  instruction  related  to  decision  making  in  complex 
domains. 

Second,  there  does  seem  to  be  a  pattern  in  which  certain  types  of  scenarios  tend  to  have  more 
complex  decision  making  challenges  than  others.  Operations,  whether  domestic  or  expeditionary, 
tend  to  have  higher  complexity  than  day-to-day  domestic  functions.  Because  of  the  limited  number 
of  scenarios  (particularly  that  there  is  only  one  domestic  operations  scenario),  it  is  difficult  to 
justify  making  many  conclusions  about  why  one  type  of  scenario  is  more  complex  than  others. 
Flowever,  it  can  be  noted  from  comparing  the  overall  complexity  scores  to  the  complexity  scores 
for  the  five  factors  individually  that  the  order  of  the  scenarios  remains  fairly  constant.  That  is,  the 
overall  complexity  rankings  of  the  scenarios  are  not  being  driven  by  one  or  two  of  the  five 
complexity  factors.  Rather,  it  appears  that  scenarios  that  tend  to  be  very  complex  tend  to  be 
complex  on  all  five  factors,  and  those  that  tend  to  be  less  complex  tend  to  be  less  complex  on  all 
five  factors,  adding  support  to  the  conclusion  that  operational  scenarios  tend  to  be  more  complex 
overall  than  domestic  day-to-day  scenarios. 

3.4. 3. 2  Bottom-up  challenges  analysis 

The  bottom-up  challenges  analysis  was  based  upon  things  that  the  SMEs  explicitly  mentioned  as 
being  challenging  about  the  decisions  they  had  to  make  in  the  scenario  they  were  describing.  The 
list  of  challenges  and  education  and  training  suggestions  were  compiled  into  a  spreadsheet  (see 
Annex  D).  Once  the  list  was  compiled,  the  challenges  were  examined  to  determine  if  there  were 
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broader  themes  and  subthemes  present.  These  themes  were  then  summarized  in  the  spreadsheet. 
Descriptions  of  the  general  themes  are  provided  below  in  Table  16,  along  with  subcomponents  of 
the  themes,  the  scenario  types  in  which  the  challenges  were  found,  and  the  number  of  challenges 
and  education  and  training  suggestions  found  related  to  the  theme.  Note  that  some  challenges  as 
stated  by  the  SMEs  had  multiple  components;  these  challenges  were  duplicated  and  entered  in 
multiple  themes  as  appropriate.  There  were  a  total  of  304  challenges  and  education  and  training 
suggestions  noted  by  the  SMEs. 


Table  16:  Bottom-up  Analysis  general  themes 


General 

Theme 

Description 

Subcomponents 

Scenario 

Types 

Represented 

Number  of 
Challenges  and 
Education/Training 
Suggestions 

Authority 

Challenges 
related  to  the 
power 
hierarchy 

Subcomponents  include  challenges  related 
to  when  people  overstep  their  authority, 
when  to  call  upon  higher  authority,  not 
having  enough  authority  to  influence  a 
situation,  and  having  to  deal  with  a  loss  of 
control  when  passing  an  issue  on  to  higher 
authority 

Domestic 
only  (i.e.,  not 
noted  in 
expeditionary 
operations) 

Some 

operational, 
some  day-to- 
day 

6 

Collaboration 

Challenges 
related  to 
having  to  work 
with  others 

Subcomponents  include  challenges  related 
to  flexibility  requirements,  having 
insufficient  lead  time,  the  requirement  to 
have  joint  training,  having  to  collaborate 
with  people  with  difficult  personalities, 
misunderstandings  about  the  roles  of 
collaborators,  a  lack  of  knowledge  among 
collaborators,  co-location  (or  lack  thereof), 
change  resistance  of  collaborators,  distrust, 
hidden  agendas,  adversarial  relationships, 
high  turnover,  low  motivation,  effects  of 
collaboration  difficulties  on  workload, 
challenges  to  credibility,  need  to  effectively 
share  information,  knowing  the  team, 
morale  and  stress,  maintaining  leadership, 
a  high  level  of  interdependence  with 
collaborators,  difficulty  mentoring, 
coordinating  work,  requiring  high  emotional 
intelligence,  requiring  good  negotiation 
skills,  having  the  proper  people  involved  in 
work,  including  people  with  diverse 
experience,  giving  credit  to  collaborators, 
and  dealing  with  collaborators’  individual 
differences 

All 

90 

Communication 

Challenges 
related  to 
exchanging 
information 

Subcomponents  include  challenges  related 
to  inadequate  communication  procedures, 
when  collaborators  use  different  jargon, 
when  communication  is  not  clear  and 

All 

17 
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General 

Theme 

Description 

Subcomponents 

Scenario 

Types 

Represented 

Number  of 
Challenges  and 
Education/Training 
Suggestions 

with  others 

honest,  inadequate  communications 
infrastructure,  when  necessary  procedures 
for  communicating  information  were  not  in 
place,  when  communication  was  vague,  not 
timely,  or  not  complete,  and  when 
communication  flow  was  more  rapid  among 
civilians  than  among  CF  personnel 

Culture 

Challenges 
related  to 
cultural 
differences 

Subcomponents  include  speaking  different 
languages,  lack  of  common  understanding, 
differences  in  the  legal  system,  corruption, 
different  beliefs  in  individual  control, 
different  social  standards,  different  literacy 
rates,  and  the  need  to  save  face 

Expeditionary 

only 

16 

Direction 

Challenges 
related  to 
understanding 
command 
intent  or  other 
instructions 

Subcomponents  include  getting  clear 
direction,  having  an  unclear  chain  of 
command,  being  asked  to  perform  an 
impossible  task,  and  requiring  flexibility, 

All 

17 

Evaluation 

Challenges 
related  to 
determining  if 
goals  have 
been 

accomplished 

Subcomponents  include  time  delays,  no 
baseline  available,  objective  assessment 
being  difficult,  empirical  evaluation  being 
required,  a  lack  of  feedback,  and 
unpredictability  of  feedback 

Domestic 

only 

Operational 
and  day-to- 
day 

9 

Experience 

Challenges 
related  to  the 
skills  and 
knowledge 
possessed  by 
DMs 

Subcomponents  include  using  SMEs  where 
possible,  having  the  right  person  in  the  right 
job,  using  intuition,  and  needing  relevant 
experience 

Domestic 

only 

Operational 
and  day-to- 
day 

5 

Goal  conflict 

Challenges 
related  to 
having  multiple 
goals  that  can 
conflict 

Subcomponents  include  considering 
multiple  factors,  achieving  multiple 
conflicting  objectives,  and  considering  long¬ 
term  effects 

All 

11 

Information 

Challenges 
related  to  the 
information 
available  to  the 
DM  (not  meant 
to  include 
challenges 
related  to 
communicating 

Subcomponents  include  dealing  with 
information  overload,  difficulty  getting 
correct  information,  difficulty  evaluating  the 
validity  of  information,  having  insufficient 
information,  dealing  with  changing 
information,  separating  interrelated  factors, 
high  workload,  using  first  principles  to 
generate  data,  collecting  new  data, 
updating  information,  needing  lessons 

All 

20 
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General 

Theme 

Description 

Subcomponents 

Scenario 

Types 

Represented 

Number  of 
Challenges  and 
Education/Trainlng 
Suggestions 

with 

individuals) 

learned,  and  requiring  real  world 
information 

Location 

Challenges 
related  to 
where  the 
scenario  took 
place 

Subcomponents  include  security 
requirements,  harsh  conditions,  and  lack  of 
flexibility 

All 

6 

Personality 

Challenges 
related  to  the 
personality  of 
the  DM 

Subcomponents  include  having  the  right 
person  in  the  right  job,  networking,  dealing 
with  authority,  effects  of  being  Type  A,  and 
handling  independence 

All 

5 

Planning 

Challenges 
related  to 
creating  or 
modifying 
plans 

Subcomponents  include  insufficient 
planning,  having  the  right  planning  tool  at 
the  right  time,  dealing  with  politics,  having  a 
lack  of  previous  similar  events,  uncertainty, 
incorrect  assumptions,  scope  changes, 
incorrect  planning,  procedure  changes,  lack 
of  required  information,  interdependence  of 
planning  components,  long  lead  times,  use 
of  trigger  points,  adapting  plans,  backup 
plans,  high  rate  of  change,  changes 
dictated  by  outside  forces,  resource 
shifting,  requiring  updated  information, 
identifying  needs  early,  requiring  flexibility, 
short  timelines,  lack  of  control,  high  stakes, 
using  available  resources,  requirement  for 
on-going  planning,  using  go/no  go  criteria, 
being  vulnerable  to  the  unexpected,  and 
unpredictability 

All 

50 

Resources 

Challenges 
related  to 
managing  or 
finding 
adequate 
resources 

Subcomponents  include  budget  concerns, 
communication  infrastructure,  power 
infrastructure,  lack  of  equipment,  timeline, 
planning,  need  to  achieve  multiple 
objectives,  having  a  funding  review, 
inconsistent  expectations,  time  limitations, 
changes  in  available  resources  and 
consequences,  uncertainty,  roads, 
transportation,  availability  in  general, 
scheduling,  balancing,  staff,  location, 
replenishment,  sleep  and  rest,  and 
information 

All 

43 

Role 

justification 

Challenges 
related  to 
having  to 
convince 
others  that  the 

No  subcomponents  noted 

Domestic 

day-to-day 

and 

expeditionary 

7 
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General 

Theme 

Description 

Subcomponents 

Scenario 

Types 

Represented 

Number  of 
Challenges  and 
Education/Training 
Suggestions 

DM’s  role  or 
the  role  of 
trainees  is 
important 

Strategic 

issues 

Challenges 
related  to  the 
strategic  level 
of  operation 

(e.g., 

governmental 

directives; 

considering 

strategic 

implications  of 

actions) 

No  subcomponents  noted 

Domestic 

day-to-day 

and 

expeditionary 

3 

There  were  several  types  of  challenges  that  did  not  appear  in  all  types  of  scenarios  (domestic 
operations,  domestic  day-to-day  functions,  and  expeditionary  operations).  Several  challenge  types 
appeared  only  in  domestic  scenarios  (“Authority”,  “Evaluation”,  and  “Experience”),  “Culture” 
challenges  appeared  only  in  expeditionary  scenarios,  and  some  themes  appeared  in  domestic  day- 
to-day  and  expeditionary  scenarios,  but  not  in  domestic  operations  (“Role  justification”  and 
“Strategic  issues”).  Because  these  challenges  did  not  appear  across  all  scenario  types,  this  may 
indicate  that  there  are  different  types  of  decision  making  challenges  that  are  found  in  different 
situations,  and  different  types  of  instruction  may  be  required.  Of  course,  it  is  also  possible  that 
these  challenges  are  present  to  an  equal  extent  in  all  scenario  types,  and  were  simply  not  detected. 

Elowever,  although  there  were  some  challenge  types  that  were  not  found  across  all  scenario  types,  a 
large  number  were  (Collaboration,  Communication,  Direction,  Goal  conflict,  Information, 

Location,  Personality,  Planning,  and  Resources).  This  suggests  that  there  are  challenges  that  make 
decision  making  difficult  across  contexts.  Therefore,  decision  making  instruction  could  be  utilized 
to  improve  how  these  challenges  are  handled  and  this  instruction  would  be  helpful  in  a  great  many 
situations.  The  importance  of  teamwork,  collaboration  and  communication  to  CF  success  has  been 
documented  in  previous  research  also  (e.g.,  see  Thomson,  Adams,  Hall,  Brown,  Flear,  2011). 

3.4. 3. 3  Bottom-up  education  and  training  suggestions 

The  bottom-up  analysis  was  examined  and  specific  education  and  training  suggestions  were 
identified  related  to  the  general  themes.  As  appropriate,  specific  suggestions  were  made  a  bit  more 
general  to  combine  closely  related  suggestions.  For  example,  “Make  sure  communication  is  clear” 
was  used  to  summarize  the  specific  SME  suggestions  “Comes  down  to  making  sure  all 
communication  is  clear”  and  “Need  to  work  on  getting  everyone  speaking  the  same  language”.  See 
Table  17  for  the  list  of  bottom-up  education  and  training  suggestions. 
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Table  17:  Bottom-up  training  suggestions 


General  Theme 

Description 

General  Education/Training  Suggestions 

Number  of 
Specific 

Education/Training 

Suggestions 

Authority 

Challenges 
related  to  the 
power  hierarchy 

Be  sure  to  engage  your  higher  authority  at  the  proper 
time 

1 

Collaboration 

Challenges 
related  to  having 
to  work  with 
others 

Bring  all  required  people  together  for  face-to-face 
meetings  to  solve  problems,  coordinate,  make 
decisions  as  a  team,  understand  people’s  views  and 
needs,  etc. 

43 

Effective  collaboration  requires  instruction  to  evolve  a 
common  understanding  between  organizations  and  to 
create  a  network  of  personal  relationships  and  build 
trust 

CF  personnel  must  have  good  emotional  intelligence, 
negotiation  skills,  interpersonal  skills,  and  be  able  to 
build  relationships 

Know  your  team  -  who  is  on  the  team,  their  level  and 
type  of  knowledge,  their  personality,  whether  they  are 
team  players 

Build  and  use  a  personal  network  of  relationships 

Coordinate  involvement  of  collaborators  -  get  buy-in, 
notify  early  when  you  want  groups  involved 

Take  care  of  group  cohesion,  keep  people  engaged 

Communication 

Challenges 
related  to 
exchanging 
information  with 
others 

Make  sure  communication  is  clear 

5 

Make  sure  you  get  all  required  information 

Culture 

Challenges 
related  to 
cultural 
differences 

Make  sure  you  understand  the  extent  of  cultural 
differences  and  their  impact  on  your  mission 

1 

Direction 

Challenges 
related  to 
understanding 
command  intent 
or  other 
instructions 

Make  sure  you  have  clear  intent  and  that  it  is  recorded 

8 

Need  to  ensure  that  the  command  structure  is  suitable 
for  the  task 

Evaluation 

Challenges 
related  to 
determining  if 
goals  have  been 

Need  to  compare  plan  to  actual  steps  taken  and 
identify  what  worked  and  whether  assumptions  were 
valid  -  this  needs  to  be  done  in  an  accessible  way  so 
that  it  is  used 

4 
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General  Theme 

Description 

General  Education/Training  Suggestions 

Number  of 
Specific 

Education/Training 

Suggestions 

accomplished 

Use  empirical  evidence  when  available  and  don’t  rely 
on  experience  and  intuition  if  you  don’t  have  to 

Experience 

Challenges 
related  to  the 
skills  and 
knowledge 
possessed  by 

DMs 

Need  the  right  person  in  the  right  job  -  proper  and 
recent  experience  -  get  people  with  the  experience 
you  need  (e.g.,  SMEs) 

5 

Goal  conflict 

Challenges 
related  to  having 
multiple  goals 
that  can  conflict 

Need  to  carefully  consider  pros  and  cons  of  possible 
decisions  with  conflicting  goals 

2 

Information 

Challenges 
related  to  the 
information 
available  to  the 

DM  (not  meant 
to  include 
challenges 
related  to 
communicating 
with  individuals) 

Should  ensure  that  you  have  access  to  recently 
updated  information 

4 

Be  careful  about  data  accuracy 

“Real  world”  information  is  often  better  than  instruction 

Location 

Challenges 
related  to  where 
the  scenario 
took  place 

No  education  or  training  suggestions  for  this 
challenge. 

0 

Personality 

Challenges 
related  to  the 
personality  of 
the  DM 

Need  to  have  the  right  personality  for  the  job  (e.g., 
how  you  handle  lack  of  authority,  how  you  handle 
independence,  ability  to  establish  relationships) 

5 

Planning 

Challenges 
related  to 
creating  or 
modifying  plans 

Need  to  be  able  to  see  when  plans  need  to  change 

11 

Need  to  be  flexible  and  able  to  handle  ambiguity  in 
planning 

Need  to  understand  how  much  planning  is  required 
(e.g.,  how  much  detail,  how  far  in  advance) 

Use  planning  techniques  as  required  (go/no  go  points, 
trigger  points,  OPP  vs.  IPP) 

Resources 

Challenges 
related  to 
managing  or 
finding  adequate 
resources 

If  possible,  try  to  take  advantage  of  related  resource 
pools  to  achieve  multiple  goals 

2 

Time  resource  acquisition  properly  relative  to  when 
resources  can  be  used 
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General  Theme 

Description 

General  Education/Training  Suggestions 

Number  of 
Specific 

Education/Training 

Suggestions 

Role  justification 

Challenges 
related  to  having 
to  convince 
others  that  the 
DM’s  role  or  the 
role  of  trainees 
is  important 

Need  to  be  able  to  justify  your  role 

4 

Strategic  issues 

Challenges 
related  to  the 
strategic  level  of 
operation  (e.g., 
governmental 
directives; 
considering 
strategic 
implications  of 
actions) 

Try  to  be  sensitive  to  potential  strategic  issues  in  all 
aspects  of  planning 

1 

As  can  be  seen  from  Table  17,  there  were  many  (96)  education  and  training  suggestions  offered  by 
the  SMEs,  and  much  overlap  (as  indicated  by  the  fact  that  usually  there  are  many  more  Specific 
suggestions  than  General  ones).  Most  suggestions  were  related  to  enhancing  collaboration,  possibly 
indicating  that  this  is  an  area  where  CF  instruction  should  be  improved. 

3.4.4  Collective  Analyses 

Mapping  the  five  main  complexity  factors  to  the  scenarios  showed  that  the  complexity  factors 
appear  to  be  relevant  to  the  experiences  of  the  SMEs.  The  collective  analyses  (overlap  and  gap 
analyses,  described  in  Section  2.4.4)  were  performed  to  integrate  the  previous  analyses  and  answer 
two  remaining  important  questions.  These  questions  were: 

1 .  What  is  the  degree  of  overlap  between  the  challenges  identified  by  SMEs  and  the  five 
main  challenges  that  we  examined  from  the  complexity  literature? 

2.  What  are  the  gaps  in  the  complexity  literature  that  have  to  be  addressed  in  CF  education 
and  training  supporting  decision  making? 

3. 4.4.1  Examining  overlap  between  complexity  research  and  SME  experience 

Once  the  bottom-up  challenges  analysis  was  performed,  the  general  themes  were  examined  to 
determine  the  extent  to  which  they  contained  components  of  the  five  main  complexity  factors 
identified  from  the  complexity  literature  review.  This  was  done  to  determine  whether  the  actual 
experience  of  CF  personnel  can  be  effectively  described  using  complexity  ideas,  or  whether 
additional  ways  of  presenting  information  about  challenges  would  be  required  to  facilitate 
education  and  training. 

To  determine  whether  the  general  theme  overlapped  with  the  complexity  category,  the  specific 
examples  related  to  each  theme  were  examined.  If  an  example  could  be  found  which  appeared  to  be 
related  to  one  of  the  five  main  complexity  factors,  it  was  provided  and  taken  as  evidence  for 
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overlap  between  the  general  theme  and  the  complexity  factor.  If  no  example  could  be  found  that 
was  obviously  related  to  one  of  the  five  complexity  factors,  then  that  general  theme  was  considered 
to  not  overlap  with  the  complexity  factor  (although  it  could  be  that  this  information  was  simply  not 
obtained  in  this  project).  The  description  of  the  overlap  between  the  two  challenge  analyses  is 
presented  in  Table  18. 


Table  18:  Overlap  between  challenge  analyses 


General 

Theme 

Description 

Subcomponents 

Overlap 

Authority 

Challenges 
related  to  the 
power 
hierarchy 

Subcomponents  include 
challenges  related  to  when 
people  overstep  their 
authority,  when  to  call  upon 
higher  authority,  not  having 
enough  authority  to  influence 
a  situation,  etc. 

Contains  elements  of  dynamics  (e.q.,  makinq  sure 
to  engage  higher  authority  at  proper  points  in 
time),  multiple  conflictinq  qoals  (e.q.,  qettinq 
resolution  by  engaging  higher  authority  vs. 
maintaining  control  of  decision  making),  and 
independent  aqents  (e.q.,  stakeholders  were 
overstepping  their  range  of  authority). 

There  were  no  obvious  examples  of  connectivity 
or  underspecified  goals. 

Collaboration 

Challenges 
related  to 
having  to  work 
with  others 

Subcomponents  include 
challenges  related  to  co- 
location  (or  lack  thereof), 
change  resistance  of 
collaborators,  distrust,  hidden 
agendas,  etc. 

Contains  elements  of  connectivity  (e.q.,  upcominq 
leaders  in  collaborating  organizations  should  be 
identified  and  invited  to  attend  CF  education  and 
training.  This  would  result  in  building  familiarity 
with  organizations,  people,  and  jargon,  create  the 
possibility  for  on-going  collaborative  exercises, 
and  create  an  embedded  liaison),  dynamics  (e.q., 
meeting  requests  were  responded  to  less 
favourably  over  time),  multiple  conflictinq  qoals 
(e.g.,  delicate  balance  between  giving  accurate 
and  helpful  feedback  and  making  people 
demotivated),  underspecified  qoals  (e.q.,  the  DM 
tried  to  meet  demands  for  requirements  but 
stakeholders  kept  coming  back  and  asking  for 
more  and  for  information  to  be  presented  in 
different  wavs),  and  independent  aqents  (e.q., 
people  were  highly  motivated  to  be  self- 
protective). 

Communication 

Challenges 
related  to 
exchanging 
information 
with  others 

Subcomponents  include  when 
collaborators  use  different 
jargon,  when  necessary 
procedures  for  communicating 
information  were  not  in  place, 
when  communication  was 
vague,  etc. 

Contains  elements  of  connectivity  (e.q.,  no  nexus 
existed  for  communicating  CF  intelligence  to  other 
organizations  (e.g.,  RCMP)  and  this  was  a  serious 
problem  as  it  was  illegal  to  communicate 
information  from  CF  assets  that  had  to  do  with 
conducting  surveillance  of  Canadians  on 

Canadian  soil),  dynamics  (e.q.,  news  can  travel 
quickly  in  the  Afghan  population,  adding  to  the  risk 
of  riots),  and  independent  aqents  (e.q.,  some  team 
members  did  not  appropriately  communicate 
information  to  the  DM  in  a  timely  way). 

There  were  no  obvious  examples  of  multiple 
conflicting  goals  or  underspecified  goals. 
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General 

Theme 

Description 

Subcomponents 

Overlap 

Culture 

Challenges 
related  to 
cultural 
differences 

Subcomponents  include 
speaking  different  languages, 
lack  of  common 
understanding,  differences  in 
the  legal  system,  etc. 

Contains  elements  of  connectivity  (e.q.,  low 
literacy  rate  in  Afghanistan  so  there  are  few 
written  records),  and  independent  aqents  (e.q., 
there  is  a  high  level  of  corruption  in  Afghanistan). 

There  were  no  obvious  examples  of  dynamics, 
multiple  conflicting  goals,  or  underspecified  goals. 

Direction 

Challenges 
related  to 
understanding 
command 
intent  or  other 
instructions 

Subcomponents  include 
getting  clear  direction,  having 
an  unclear  chain  of  command, 
being  asked  to  perform  an 
impossible  task,  etc. 

Contains  elements  of  connectivity  (e.q.,  need  to 
coordinate  as  much  as  possible  with  higher 
command  (e.g.,  CANOSCOM)  to  get  information 
needed  to  form  education  and  training  objectives), 
multiple  conflictinq  qoals  (e.q.,  qiven  instructions 
from  higher  command  to  do  things  without  any 
funds  available),  underspecified  qoals  (e.q.,  can 
be  a  problem  that  Generals  don’t  give  very 
concrete  and  clear  intent),  and  independent 
aqents  (e.q.,  requests  for  clarification  from 
superiors  about  prioritization  of  programs  did  not 
result  in  clear  direction). 

There  were  no  obvious  examples  of  dynamics. 

Evaluation 

Challenges 
related  to 
determining  if 
goals  have 
been 

accomplished 

Subcomponents  include  time 
delays,  no  baseline  available, 
objective  assessment  being 
difficult,  etc. 

Contains  elements  of  connectivity  (e.q.,  need  to 
analyse  steps  taken,  OPP  used,  compare  plans  to 
actual  operation,  identify  what  worked,  examine 
assumptions),  dynamics  (e.q.,  lack  of  timelv 
feedback),  and  independent  aqents  (e.q.,  trainees 
respond  unpredictably  to  events). 

There  were  no  obvious  examples  of  multiple 
conflicting  goals  or  underspecified  goals. 

Experience 

Challenges 
related  to  the 
skills  and 
knowledge 
possessed  by 
DMs 

Subcomponents  include 
making  sure  the  right  person 
is  in  the  right  job,  using 
intuition,  needing  relevant 
experience,  etc. 

Contains  elements  of  connectivity  (e.q.,  need  to 
know  how  Ottawa  works,  dealing  with  public 
servants,  procurement,  etc.),  multiple  conflictinq 
qoals  (e.g.,  need  to  be  able  to  “walk  both  sides  of 
the  fence”),  and  independent  aqents  (e.q.,  ensure 
that  people  creating  exercises  have  proper  and 
recent  experience). 

There  were  no  obvious  examples  of  dynamics  or 
underspecified  goals. 

Goal  conflict 

Challenges 
related  to 
having  multiple 
goals  that  can 
conflict 

Subcomponents  include 
considering  multiple  factors 
and  achieving  multiple 
conflicting  objectives 

Contains  elements  of  connectivity  (e.q.,  users 
have  multiple  conflicting  needs;  making  changes 
will  almost  inevitably  affect  multiple  needs  both 
positively  and  neqativelv),  dynamics  (e.q.,  have  to 
balance  short-term  and  lonq-term  qoals),  multiple 
conflictinq  qoals  (e.q.,  delicate  balance  between 
giving  accurate  and  helpful  feedback  and  making 
people  demotivated),  and  independent  aqents 
(e.g.,  changes  to  programs  should  involve 
consultation  with  a  large  number  of  stakeholders, 
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General 

Theme 

Description 

Subcomponents 

Overlap 

(e.g.,  all  elements). 

There  were  no  obvious  examples  of 
underspecified  goals. 

Information 

Challenges 
related  to  the 
information 
available  to  the 
DM  (not  meant 
to  include 
challenges 
related  to 
communicating 
with 

individuals) 

Subcomponents  include 
getting  information  with  errors, 
finding  it  difficult  to  get  correct 
information,  having  insufficient 
data,  etc. 

Contains  elements  of  connectivity  (e.q.,  proposals 
for  funding  involve  many  interrelated  factors), 
dynamics  (e.q.,  unpredictable  response  after 
information  exchange  (silence  or  many  more 
requests)),  and  independent  aqents  (e.q.,  there 
was  resistance  to  including  testing  for  all  important 
interacting  factors). 

There  were  no  obvious  examples  of  multiple 
conflicting  goals  or  underspecified  goals. 

Location 

Challenges 
related  to 
where  the 
scenario  took 
place 

Subcomponents  include  harsh 
conditions,  security 
requirements,  lack  of 
flexibility,  etc. 

Contains  elements  of  connectivity  (e.q.,  the 
environmental  effects  make  maintenance  more 
frequent),  dynamics  (e.q.,  location  of  conflict  had 
large  impact  on  flexibility  to  replenish  resources), 
and  independent  aqents  (e.q.,  demand  bv  the 
training  course  location  to  not  have  trainees  in 
uniform). 

There  were  no  obvious  examples  of  multiple 
conflicting  goals  or  underspecified  goals. 

Personality 

Challenges 
related  to  the 
personality  of 
the  DM 

Subcomponents  include 
independence,  type  A,  having 
the  right  personality  for  the 
job,  etc. 

Contains  elements  of  connectivity  (e.q.,  need  to 
establish  relationships),  and  independent  aqents 
(e.g.,  can’t  have  type-A  personalities  in  full-time 

CoE  positions  because  they  may  butt  heads  with 
the  military). 

There  were  no  obvious  examples  of  dynamics, 
multiple  conflicting  goals,  or  underspecified  goals. 

Planning 

Challenges 
related  to 
creating  or 
modifying 
plans 

Subcomponents  include 
challenging  assumptions, 
insufficient  planning,  lack  of 
control,  etc. 

Contains  elements  of  connectivity  (e.q.,  after  the 
prison  break  there  was  a  distrust  of  prison  staff 
and  many  assumptions  were  reassessed), 
dynamics  (e.q.,  traininq  cannot  relv  on  templates 
because  the  rate  of  chanqe  is  too  hiqh),  multiple 
conflictinq  qoals  (strateqies  for  controllinq  number 
of  bidders  focussed  on  getting  more  bidders  so  in 
the  end  there  were  too  manv),  underspecified 
qoals  (e.g.,  deliverables  kept  changing  so 
contractinq  was  difficult),  and  independent  aqents 
(true  sustainment  was  not  practiced  by  the  BG, 
leading  to  lack  of  planning  information). 

Resources 

Challenges 
related  to 
managing  or 
finding 

Subcomponents  include 
communications 
infrastructure,  roads,  budget, 

Contains  elements  of  connectivity  (e.q.,  difficult  to 
get  travellers  back  to  the  compound  due  to  lack  of 
vehicles  and  poor  passabilitv  of  roads),  dynamics 
(e.g.,  because  of  a  short  planning  timeline 
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General 

Theme 

Description 

Subcomponents 

Overlap 

adequate 

resources 

etc. 

changes  in  one  resource  meant  that  other 
changes  had  to  be  made  to  accommodate  it), 
multiple  conflicting  qoals  (e.q.,  requirements  for 
new  training  programs  are  given  without  additional 
resources),  underspecified  qoals  (e.q., 
disagreements  between  stakeholders  regarding 
scope  of  fundinq),  and  independent  aqents  (e.q., 
the  time  of  trainees  is  limited,  so  training  package 
size  is  limited). 

Role 

justification 

Challenges 
related  to 
having  to 
convince 
others  that  the 
DM’s  role  or 
the  role  of 
trainees  is 
important 

No  subcomponents 

Contains  elements  of  connectivity  (e.q.,  an  outside 
consultant  might  have  had  a  bigger  impact), 
underspecified  qoals  (e.q.,  “qo  PSYOPS  those 
quvs”),  and  independent  aqents  (e.q.,  lack  of 
appreciation  for  DM’s  contribution). 

There  were  no  obvious  examples  of  dynamics  or 
multiple  conflicting  goals. 

Strategic  issues 

Challenges 
related  to  the 
strategic  level 
of  operation 

No  subcomponents 

Contains  elements  of  connectivity  (e.q., 
government  announcements  can  force 
unexpected  readjustments  in  traininq),  multiple 
conflicting  qoals  (e.q.,  political/strateqic  concerns 
limited  resource  options),  and  independent  aqents 
(e.g.,  got  unwanted  attention  at  strategic  level 
when  wore  military  uniforms). 

There  were  no  obvious  examples  of  dynamics  or 
underspecified  goals. 

As  can  be  seen  from  Table  18,  most  of  the  general  themes  from  the  bottom-up  challenges  analysis 
have  significant  overlap  with  the  five  main  complexity  factors  examined  in  this  project.  However, 
although  there  is  significant  overlap,  it  should  be  noted  that  only  in  the  case  of  the  general  theme 
“Goal  conflict”  and  the  complexity  factor  “Multiple  conflicting  goals”  does  there  appear  to  be  a 
close  connection  between  how  the  SMEs  think  of  the  challenges  to  their  decision  making  and  the 
way  that  these  challenges  are  presented  in  the  complexity  literature.  This  is  important  for 
education,  as  CF  personnel  should  be  able  to  identify  with  the  way  that  challenges  are  presented 
during  education  and  training  to  maximize  understanding  of  the  ideas  being  presented. 

Another  way  of  looking  at  the  data  presented  in  Table  18  is  to  organize  the  data  according  to  the 
five  complexity  factors.  This  has  been  done  in  Table  19  below. 
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Table  19:  Themes  organized  by  five  complexity  factors 


Complexity  Factor 

General  Themes  which  Contain  the  Complexity  Factor 

Connectivity 

Collaboration;  Communication;  Culture;  Direction;  Evaluation;  Experience;  Goal  conflict; 
Information;  Location;  Personality;  Planning;  Resources;  Role  justification;  Strategic  issues 

Dynamics 

Authority;  Collaboration;  Communication;  Evaluation;  Goal  conflict;  Information;  Location; 
Planning;  Resources 

Multiple  conflicting 
goals 

Authority;  Collaboration;  Direction;  Experience;  Goal  conflict;  Planning;  Resources;  Strategic 
issues 

Underspecified  goals 

Collaboration;  Direction;  Planning;  Resources;  Role  justification 

Independent  agents 

Authority;  Collaboration;  Communication;  Culture;  Direction;  Evaluation;  Experience;  Goal 
conflict;  Information;  Location;  Personality;  Planning;  Resources;  Role  justification;  Strategic 
issues 

3.4. 4. 2  Gap  analysis 

One  of  the  important  pieces  of  information  required  from  this  comparison  of  the  complexity 
literature  and  the  actual  experience  of  CF  personnel  is  a  determination  of  what  important 
challenges  to  the  decision  making  of  CF  personnel  in  seemingly  complex  environments  are  not 
well  captured  in  the  complexity  literature.  These  gaps  need  to  be  identified  and  investigated  further 
if  the  CF  education  and  training  processes  are  to  be  adequately  supported.  Note  that  the  gap 
analysis  is  based  upon  the  literature  review  described  earlier  in  this  report,  and  therefore  is  limited 
to  the  findings  described  in  that  subsection  of  documents  which  were  reviewed. 

To  determine  if  there  were  gaps,  the  general  themes  and  related  examples  generated  from  the 
bottom-up  challenges  analysis  were  examined  and  anything  not  apparently  represented  in  the 
complexity  literature  was  identified  and  included  in  the  gap  analysis.  Explicit  examples  of  how  the 
complexity  concepts  have  been  investigated  in  microworlds  and  potential  limitations  in  this 
research  are  included  where  applicable;  however,  just  because  a  gap  is  noted  does  not  mean  that  it 
would  be  difficult  to  investigate  these  concepts.  The  gap  analysis  is  presented  in  Table  20  below: 


Table  20:  Gap  analysis 


General  Theme 

Description 

Subcomponents 

Gap  Analysis 

Authority 

Challenges  related  to 
the  power  hierarchy 

Subcomponents 
include  challenges 
related  to  when  people 
overstep  their  authority, 
when  to  call  upon 
higher  authority,  not 
having  enough 
authority  to  influence  a 
situation,  etc. 

The  complexity  literature  mentions 
independent  agents;  it  is  possible 
that  some  of  these  independent 
agents  could  be  in  a  position  of 
authority  over  the  DM,  although 
this  was  not  explored  in  the 
literature  reviewed. 

Often  in  microworlds  the 
participant  is  the  only  DM;  this 
means  they  are  the  main  decision 
making  authority  in  the  situation 
and  do  not  have  to  worry  about  a 
higher  authority.  Team  decision 
making  is  likely  underrepresented 
and  more  research  is  needed  to 
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General  Theme 

Description 

Subcomponents 

Gap  Analysis 

understand  how  authority 
influences  complex  decision 
making. 

There  are  examples  in  CF 
experience  where  the  DM  can  only 
advise  and  doesn’t  have  overt 
decision  making  power. 

There  is  also  an  example  in  CF 
experience  of  the  rank  of  the  DM 
creating  a  perception  of  the 
importance  of  his  role  -  the 
perception  of  authority  generally  is 
not  investigated  in  microworlds. 

Collaboration 

Challenges  related  to 
having  to  work  with 
others 

Subcomponents 
include  challenges 
related  to  co-location 
(or  lack  thereof), 
change  resistance  of 
collaborators,  distrust, 
hidden  agendas,  etc. 

There  was  little  acknowledgement 
in  the  complexity  literature 
reviewed  that  decision  making  is  a 
collaborative  process,  particularly 
in  cases  where  the  entire  team 
makes  decisions  together,  rather 
than  each  team  member  being 
responsible  for  task  components. 

The  collaborative  nature  of 
decision  making  is  usually  ignored 
in  microworlds,  whereas  it  is 
critical  in  the  CF  (people  get 
direction  from  higher  command, 
they  work  in  teams,  and  decisions 
are  usually  implemented  by 
teams). 

Specific  aspects  of  collaboration 
worthy  of  mention  include:  the 
importance  of  networking,  hidden 
agendas,  co-location,  building  a 
common  understanding, 
negotiation,  having  to  deal  with 
disagreements,  team  building,  and 
dealing  with  personnel  turnover. 

Communication 

Challenges  related  to 
exchanging 
information  with 
others 

Subcomponents 
include  when 
collaborators  use 
different  jargon,  when 
necessary  procedures 
for  communicating 
information  were  not  in 
place,  when 
communication  was 
vague,  etc. 

The  need  to  gather  information 
appears  to  be  included  in  the 
complexity  literature;  however, 
there  is  little  discussion  of  how  this 
is  actually  accomplished  or  the 
challenges  involved. 

Communication  appears  to  be 
much  more  important  in  CF 
contexts  than  as  represented  in 
microworlds,  at  least  the  ones 
reviewed  in  this  project  (although 
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General  Theme 

Description 

Subcomponents 

Gap  Analysis 

there  is  a  teamwork-based 
literature  that  may  involve  this 
issue).  While  communication  plays 
a  role  in  microworlds  in  that  the 

DM  needs  to  acquire  information, 
generally  how  the  DM  interacts 
with  the  environment  is  not  framed 
in  terms  of  communication. 

Specific  aspects  of  communication 
worthy  of  note  include  the  need  for 
communication  to  be  clear, 
complete,  and  accurate,  and  the 
need  to  deal  with  different  jargon 
when  trying  to  communicate  with 
different  organizations. 

Culture 

Challenges  related  to 
cultural  differences 

Subcomponents 
include  speaking 
different  languages, 
lack  of  common 
understanding, 
differences  in  the  legal 
system,  etc. 

The  complexity  literature  mentions 
independent  agents;  it  is  possible 
that  cultural  differences  between 
the  DM  and  independent  agents 
might  add  to  the  complexity  of 
decision  making,  although  this  was 
not  explored  in  the  literature 
reviewed. 

Cultural  issues  as  a  whole  are 
generally  critical  for  the  CF  in 
expeditionary  operations  and  yet 
examples  of  them  in  microworlds 
were  not  noted,  at  least  in  the 
literature  reviewed  for  this  report. 
Important  aspects  of  culture 
include  language  differences, 
general  differences  in 
understanding  (e.g.,  what  does 
having  a  job  mean?  How  should 
you  be  trained  for  a  job?), 
differences  in  social  norms  (e.g., 
relationship  of  age  to  authority), 
social  infrastructure  (e.g.,  no 
banks  in  Afghanistan),  and 
different  literacy  rates. 

Direction 

Challenges  related  to 
understanding 
command  intent  or 
other  instructions 

Subcomponents 
include  getting  clear 
direction,  having  an 
unclear  chain  of 
command,  being  asked 
to  perform  an 
impossible  task,  etc. 

This  category  appears  to  be 
similar  to  the  fact  that  in  the 
complexity  literature  it  is  noted  that 
it  is  sometimes  difficult  to  get  clear 
goals,  whether  it  is  getting  goals 
from  higher  command  or  the  DM 
creating  concrete  goals. 

Components  of  decision  making 
involving  direction  which  are 
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important  to  the  CF  which  are  not 
stressed  in  microworlds  include 
the  need  to  obey  higher  command, 
having  the  option  to  clarify  orders 
with  higher  command,  and  the 
need  to  understand  the  command 
hierarchy. 

Evaluation 

Challenges  related  to 
determining  if  goals 
have  been 
accomplished 

Subcomponents 
include  time  delays,  no 
baseline  available, 
objective  assessment 
being  difficult,  etc. 

This  theme  overlaps  heavily  with 
the  fact  that  the  complexity 
literature  notes  that  it  is  often 
difficult  to  determine  your  net 
effect  on  the  environment,  that 
feedback  is  delayed,  etc. 

Microworld  researchers  are  trying 
to  improve  how  people  evaluate 
their  impact  on  complex 
environments  -  though  clear 
solutions  have  yet  to  be 
determined.  Thus,  there  are  no 
obvious  gaps  between  this  theme 
and  the  complexity  literature. 

Experience 

Challenges  related  to 
the  skills  and 
knowledge 
possessed  by  DMs 

Subcomponents 
include  making  sure  the 
right  person  is  in  the 
right  job,  using  intuition, 
needing  relevant 
experience,  etc. 

Many  researchers  (e.g.,  Dorner, 
1996)  note  that  experience  likely 
plays  an  important  role  in 
determining  the  effectiveness  of  a 
DM  in  complex  decision  making. 
However,  the  actual  components 
of  experience  which  make  a 
difference  are  not  thoroughly 
understood  (e.g.,  are  there  general 
skills  or  situation  specific  skills?). 

Often  microworlds  use  novice 
decision  makers  (at  least,  the  DM 
has  limited  experience  in  that 
environment).  In  the  CF,  people 
are  trained  to  fill  their  roles,  and 
are  often  carefully  selected  for 
their  skills  and  experience  (“need 
the  right  person  in  the  right  job”). 

Goal  conflict 

Challenges  related  to 
having  multiple  goals 
that  can  conflict 

Subcomponents 
include  considering 
multiple  factors  and 
achieving  multiple 
conflicting  objectives 

This  theme  appears  to  overlap 
completely  with  the  factor  “Multiple 
conflicting  goals”  from  the 
complexity  literature. 

Some  specific  examples  of  goal 
conflict  have  social  aspects  which 
are  usually  not  included  in 
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microworlds  (e.g.,  managing 
stakeholder  expectations)2. 

Information 

Challenges  related  to 
the  information 
available  to  the  DM 
(not  meant  to  include 
challenges  related  to 
communicating  with 
individuals) 

Subcomponents 
include  getting 
information  with  errors, 
finding  it  difficult  to  get 
correct  information, 
having  insufficient  data, 
etc. 

This  theme  overlaps  heavily  with 
the  idea  in  the  complexity  literature 
that  it  is  difficult  to  determine  your 
net  effect  on  the  environment,  and 
that  there  is  often  a  great  deal  of 
information  to  deal  with  and  it  may 
change  over  time,  etc.  One 
noteworthy  aspect  of  this  theme  is 
that  information  can  actually  be 
incorrect. 

Location 

Challenges  related  to 
where  the  scenario 
took  place 

Subcomponents 
include  harsh 
conditions,  security 
requirements,  lack  of 
flexibility,  etc. 

The  complexity  literature  generally 
includes  consideration  that  factors 
related  to  the  location  involved  in 
decision  making  (e.g.,  weather) 
might  make  a  decision  more 
complex,  even  if  they  do  not 
specify  a  location  per  se. 

Personality 

Challenges  related  to 
the  personality  of  the 
DM 

Subcomponents 
include  independence, 
type  A,  having  the  right 
personality  for  the  job, 
etc. 

The  complexity  literature  does 
consider  individual  differences, 
although  these  are  generally 
differences  in  experience  and 
general  decision  making  skill 
rather  than  personality.  Some  of 
the  behaviours  noted  as  being 
effective  or  ineffective  in  complex 
decision  making  contexts  (e.g., 
Grisogono,  2010)  could  be  framed 
in  terms  of  personality  (e.g.,  the 
tendency  to  attribute  failures  to 
oneself  or  others). 

The  personality  of  the  DM  is  not 
generally  considered  in  microworld 
research,  although  some  is 
considered  in  the  context  of 
determining  whether  there  are 
good  and  poor  actor  behaviours 
related  to  decision  making.  Most 
aspects  of  personality  mentioned 
by  the  CF  personnel  do  not  seem 
to  appear  in  the  complexity 
literature  (e.g.,  being  type  A,  being 

"  Additional  research  from  cognitive  psychology  and  cognitive  science  (e.g.,  goal  formation  and  selection) 
may  also  provide  valuable  insights  that  could  be  incorporated  into  complex  decision  making  and  microworld 
studies. 
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able  to  get  along  with  other  people 
and  network). 

As  noted  in  the  section  discussion 
the  results  of  the  literature  review 
(see  Section  3.1),  there  is  currently 
an  attempt  by  Dorner  to  create  a 
framework  integrating  personality 
and  motivational  elements  with 
good/poor  actor  behaviours. 
Unfortunately,  as  of  the  writing  of 
this  report  that  literature  is  not 
readily  available  in  English,  and  so 
is  not  cited  or  reviewed  here. 

Planning 

Challenges  related  to 
creating  or  modifying 
plans 

Subcomponents 
include  challenging 
assumptions, 
insufficient  planning, 
lack  of  control,  etc. 

There  appear  to  be  a  lot  of 
complexity  factors  that  are  relevant 
for  this  theme  (which  is  quite 
large).  The  need  to  plan  and 
modify  plans  is  central  in  the 
complexity  literature. 

Components  of  planning  that  might 
not  be  adequately  covered  by  the 
complexity  literature  include  the 
need  to  involve  other  groups  in 
planning  as  well  as  understand 
how  the  plan  will  be  perceived  by 
observers  (similar  to  the 
Collaboration  theme). 

Resources 

Challenges  related  to 
managing  or  finding 
adequate  resources 

Subcomponents 

include 

communications 
infrastructure,  roads, 
budget,  etc. 

The  complexity  literature 
discusses  time  as  a  resource  that 
must  be  managed,  particularly  in 
terms  of  the  dynamics  of  the 
environment  requiring  the  DM  to 
monitor  time  and  interact  with  the 
environment  at  particular  times. 

Resources  typically  represent 
limitations  that  have  to  be 
considered  when  making 
decisions,  and  similar  resource 
issues  would  likely  be  used  in 
most  microworld  environments. 

Thus,  resource  management  is 
central  to  most  microworld 
research. 

One  other  aspect  of  resource 
management  -  stakeholder 
disagreements  and  the 
perceptions  related  to  resource 
allocation  -  are  important  in  CF 
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contexts  but  may  not  be 
adequately  represented  in  the 
complexity  literature. 

Role  justification 

Challenges  related  to 
having  to  convince 
others  that  the  DM’s 
role  or  the  role  of 
trainees  is  important 

No  subcomponents 

The  complexity  literature  and 
microworld  research  do  not  appear 
to  deal  with  issues  such  as  the  DM 
having  to  justify  their  role. 

Strategic  issues  (e.g., 

governmental 

directives; 

considering  strategic 
implications  of 
actions) 

Challenges  related  to 
the  strategic  level  of 
operation 

No  subcomponents 

Strategic  issues  are  extremely 
important  issues  to  the  CF  as  they 
are  generally  related  to  high  level 
international  relations.  These  types 
of  issues  could  likely  be 
represented  in  the  complexity 
literature  as  important  goals, 
although  the  term  strategic  would 
not  be  used  per  se. 

Strategic  issues  could  be  one  level 
of  factor  used  in  microworlds,  if 
they  were  presented  in  a  military 
context.  Similar  to  resource  issues. 

The  gap  analysis  presented  above  shows  that,  while  a  number  of  the  general  themes  from  the 
bottom-up  challenges  analysis  are  somewhat  similar  to  those  ideas  we  examined  from  the 
complexity  literature,  there  appear  to  be  a  number  of  gaps  in  the  complexity  literature  that  was 
reviewed  for  this  project  when  it  is  compared  to  the  experience  of  CF  SMEs.  For  example,  as 
mentioned  in  the  literature  review  section  (Section  3.1),  reviewing  team  decision  making  was 
considered  beyond  the  scope  of  this  project.  Therefore,  in  the  literature  reviewed,  there  appeared  to 
be  little  acknowledgement  in  the  complexity  literature  that  decision  making  is  often  a  collaborative 
process,  and  DMs  rely  on  other  people  for  information  as  well  as  to  implement  their  decisions. 
Specific  related  issues  that  should  be  examined  in  future  work  (including  a  more  in-depth  literature 
review)  include  having  to  deal  with  team  building,  having  to  deal  with  personnel  turnover,  and 
having  to  negotiate.  Similarly,  the  need  for  DMs  to  justify  their  role  also  does  not  appear  to  be 
dealt  with  in  the  complexity  literature  reviewed  for  this  project,  but  may  be  addressed  elsewhere. 

As  previously  noted,  because  the  literature  review  undertaken  in  this  project  was  rather  limited,  it  is 
certainly  possible  that  the  gaps  identified  in  this  analysis  are  simply  examined  in  other  research  that 
was  not  included  in  this  review.  Flowever,  to  the  extent  that  these  gaps  are  real  and  not  addressed  in 
the  complexity  literature,  they  present  a  risk  to  the  plan  of  using  the  complexity  literature  to  guide 
the  education  and  training  of  military  SMEs  to  make  decisions  in  complex  environments,  as  critical 
factors  may  be  omitted. 
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The  overall  goal  of  this  project  was  to  determine  the  usefulness  of  existing  research  in  complex 
decision  making  (in  particular,  findings  from  research  using  microworlds)  for  enhancing  military 
education  and  training  related  to  decision  making  in  complex  environments.  To  accomplish  this 
goal,  we  attempted  to  answer  several  questions,  including: 

1.  Do  CF  personnel  experience  the  types  of  decision  making  challenges  described  in  the 
complexity  literature? 

2.  Are  there  challenges  that  CF  personnel  typically  face  in  their  decision  making  that  are  not 
covered  by  the  complexity  literature? 

3.  What  challenges  need  to  be  addressed  in  CF  education  and  training  to  support  decision 
making? 

4.  Can  microworlds  likely  be  used  to  facilitate  this  education  and  training? 

These  questions  will  be  answered  in  turn.  As  well,  in  this  section  we  discuss  additional  insights 
gained,  limitations  of  this  research,  and  possible  future  work. 

4.1  Do  CF  personnel  experience  the  types  of  decision  making 
challenges  described  in  the  complexity  literature? 

CF  personnel  do  seem  to  experience  the  types  of  decision  making  challenges  described  in  the 
complexity  literature.  Anecdotally,  when  we  reviewed  the  five  factors  with  the  SMEs,  they  had 
little  trouble  coming  up  with  multiple  examples  of  each  of  the  five  factors  from  their  experience, 
and  generally  agreed  that  such  factors  are  challenging  for  their  decision  making. 

We  empirically  assessed  whether  there  was  an  overlap  between  the  five  main  complexity  factors 
we  were  examining  and  the  experience  of  the  CF  SMEs  by  mapping  the  five  complexity  factors  to 
the  scenarios  described  by  the  SMEs.  In  every  case,  the  five  factors  were  represented  in  the 
scenarios  to  at  least  some  extent,  and  added  to  the  difficulty  of  the  scenario.  Thus,  it  can  be 
concluded  that  the  challenges  described  in  the  complexity  literature  are  experienced  by  CF  SMEs. 

4.2  Are  there  challenges  that  CF  personnel  typically  face  in  their 
decision  making  that  are  not  covered  by  the  complexity 
literature? 

There  do  appear  to  be  many  types  of  decision  making  challenges  related  to  complex  environments 
that  are  not  adequately  addressed  in  the  complexity  literature.  The  gap  analysis  identified  many 
challenges  experienced  by  CF  personnel  which  were  not  identified  in  the  complexity  literature 
review.  Some  aspects  of  CF  decision  making  which  can  be  challenging  are  either  not  present  in  the 
literature  or  deemed  to  be  beyond  the  scope  of  the  literature  review  performed  for  this  project  (e.g., 
the  collaborative  nature  of  decision  making,  role  justification),  whereas  some  challenges  have 
important  components  which  were  not  included  in  the  literature  review  (e.g.,  communication  issues 
such  as  jargon  differences  between  organizations).  As  noted,  it  may  be  that  these  issues  are 
addressed  in  other  complexity  research  which  was  not  examined  in  the  current  project. 
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4.3  What  challenges  need  to  be  addressed  in  CF  education  and 
training  to  support  decision  making? 

The  first  way  to  identify  challenges  that  need  to  be  addressed  in  CF  education  and  training  to 
support  decision  making  is  to  examine  the  education  and  training  suggestions  offered  by  the  SMEs. 
A  second  way  to  identify  challenges  that  need  to  be  addressed  in  CF  education  and  training  is  to 
examine  the  challenges  identified  from  the  top-down  analysis,  and  identify  challenge  types  that 
may  be  the  most  broadly  applicable  and  therefore  good  targets  for  intervention. 

4.3.1  Challenges  from  SME  suggestions 

By  examining  the  education  and  training  suggestions  provided  by  the  SMEs,  it  can  be  seen  that,  by 
a  wide  margin,  the  largest  category  of  suggestions  had  to  do  with  enhancing  collaboration  (almost 
half  of  all  suggestions).  This  strongly  indicates  that  this  is  an  area  where  CF  instruction  should  be 
enhanced.  Key  components  related  to  enhancing  collaboration  include  creating  and  building 
networks,  making  sure  to  have  face-to-face  meetings  as  much  as  possible,  knowing  the  people  you 
are  collaborating  with  (personalities,  experience,  etc.),  and  building  skills  such  as  emotional 
intelligence,  inteipersonal  skills,  and  relationship  building.  Many  of  the  SMEs  interviewed  for  the 
project  indicated  that  these  skills  are  critical  for  success  in  many  of  the  situations  in  which  the  CF 
currently  finds  itself  operating.  As  the  CF  typically  makes  and  implements  decisions  in  a  team 
environment,  the  ability  to  effectively  collaborate  is  a  critical  decision  making  factor. 

The  education  and  training  suggestion  that  CF  personal  need  to  be  able  to  justify  their  role  (related 
to  the  Role  justification  theme),  at  least  as  presented  by  the  SMEs  we  interviewed,  does  appear  to 
indicate  that  this  is  an  additional  gap  in  current  CF  instruction.  CF  personnel  need  to  be  able  to 
justify  their  general  role  as  well  as  the  specific  role  they  have  in  a  particular  situation,  and  this 
should  be  explicitly  taught. 

Some  of  the  education  and  training  suggestions  made  by  the  SMEs  are  likely  related  to  things  that 
are  already  part  of  CF  instruction,  but  the  fact  that  they  were  included  as  challenges  and  education 
and  training  suggestions  indicates  that  they  may  not  be  performed  effectively  at  all  times.  When 
these  components  of  complex  decision  making  are  not  performed  effectively  it  likely  leads  to 
significant  challenges  with  decision  making.  Examples  include  education  and  training  suggestions 
related  to  Planning  (e.g.,  “Need  to  be  able  to  see  when  plans  need  to  change”)  and  Communication 
(e.g.,  “Make  sure  communication  is  clear”).  It  does  appear  that  these  aspects  are  taught; 
nonetheless,  there  is  some  component  of  these  decision-making  behaviours  which  are  challenging. 
This  could  be  because  they  are  difficult  to  teach,  it  is  difficult  to  understand  when  they  should  be 
performed,  it  is  difficult  to  implement  them  effectively,  etc. 

Several  education  and  training  suggestions  actually  do  not  appear  to  be  suggestions  for  enhancing 
learning;  rather,  they  appear  to  be  selection  criteria.  For  example,  making  sure  that  you  have  the 
right  person  in  the  right  job  (a  suggestion  related  to  the  Experience  theme)  and  making  sure  that 
you  have  the  right  personality  for  the  job  (a  suggestion  related  to  the  Personality  theme)  seem  to  be 
suggestions  that  certain  people  should  be  chosen  for  certain  jobs,  and  other  people  should  not  be 
chosen  for  those  jobs.  Thus,  it  is  possible  that,  in  addition  to  teaching  individuals  to  be  better  DMs, 
certain  people  may  have  certain  attributes  that  make  them  more  or  less  suitable  for  particular  role  - 
making  some  individuals  more  suitable  for  particular  complex  decision  making  environments  than 
others. 

By  examining  the  bottom-up  challenges,  it  can  be  seen  that  there  were  several  challenge  themes 
that  represented  a  large  proportion  of  the  challenges  mentioned  by  the  SMEs.  Of  3 1 0  challenges  or 
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education  and  training  suggestions  mentioned.  Collaboration  (91  challenges),  Planning  (50 
challenges),  and  Resources  (45  challenges)  represent  two  thirds  of  all  challenges.  If  limited 
instructional  changes  can  be  made,  it  would  seem  to  be  beneficial  to  focus  on  those  categories.  In 
addition  to  these  being  the  most  frequent  types  of  challenges  mentioned,  it  should  also  be  noted 
that  they  apply  across  all  three  scenario  types  examined  (domestic  operations,  domestic  day-to-day, 
and  expeditionary  operations),  which  should  also  mean  that  they  are  most  broadly  applicable  and 
addressing  them  should  provide  greater  benefit  than  addressing  challenges  that  are  only  present  in 
certain  scenario  types. 

4.3.2  Challenges  from  complexity  literature 

There  was  evidence  provided  in  the  top-down  analysis  that  all  five  complexity  factors 
(connectivity,  dynamics,  multiple  conflicting  goals,  underspecified  goals,  and  independent  agents) 
played  some  role  in  each  of  the  scenarios.  So,  there  is  some  evidence  that  these  issues  should  be 
addressed  in  CF  education  and  training  if  decision  making  is  to  be  adequately  taught.  However,  it 
would  also  be  useful  to  determine  if  there  are  patterns  in  the  degree  to  which  different  scenario 
types  are  affected  by  the  different  types  of  challenges  to  see  if  targeted  intervention  is  required,  or 
whether  there  are  challenges  that  broadly  apply  so  that  general  instruction  could  impact  a  wide 
variety  of  decision  making  situations. 

As  noted  in  the  top-down  analysis,  a  pattern  related  to  which  types  of  scenarios  tend  to  have  higher 
complexity  ratings  appears.  Briefly,  operations  tend  to  have  higher  complexity  ratings  than 
domestic  day-to-day  functions.  The  scenario  with  the  highest  complexity  score  was  the  domestic 
operations  scenario;  however,  it  is  difficult  to  make  conclusions  that  domestic  operations  are 
intrinsically  more  complex  than  expeditionary  operations,  because  there  was  only  one  example  of  a 
domestic  operation.  Thus,  it  is  possible  that  education  and  training  interventions  aimed  at 
improving  complex  decision  making  should  be  targeted  first  at  CF  personnel  with  an  operations 
focus,  rather  than  domestic  day-to-day  focus.  Whether  complex  operational  decision  making 
education  and  training  can  be  supplied  “just  in  time”  for  operational  deployment  or  is  best  provided 
throughout  the  career  of  CF  personnel  remains  unknown  to  the  authors. 

It  is  difficult  to  make  definitive  conclusions  about  the  relative  difficulty  produced  by  the  different 
complexity  factors  in  this  research,  as  the  complexity  factors  were  only  compared  within  the  set  of 
10  scenarios  produced  for  this  project.  Thus,  it  is  not  possible  to  determine  whether  one  factor 
should  be  addressed  more  than  another  factor.  However,  future  work  should  be  performed  to 
investigate  this  issue. 

4.4  Can  microworlds  likely  be  used  to  facilitate  this  education  and 
training? 

Based  on  the  gap  analysis,  there  are  many  factors  which  are  important  for  CF  decision  making  that 
have  not  been  implemented  in  microworlds;  some  of  these  could  probably  be  easily  implemented 
(e.g.,  DMs  getting  incorrect  information),  but  some  would  likely  be  much  more  difficult  to 
implement  successfully  (e.g.,  training  people  how  to  build  relationships  and  increase  their 
interpersonal  skills). 

The  largest  challenge  area  pointed  out  by  the  SMEs  as  needing  development  was  the  ability  to 
facilitate  collaboration.  The  typical  microworld  setting  where  a  single  DM  interacts  with  a 
computerized  virtual  world  seems  fundamentally  inadequate  for  addressing  the  issues  related  to 
facilitating  collaboration,  as  the  DM  is  not  interacting  with  real  people,  building  a  real  network  of 
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relationships,  etc.  While  microworld  environments  have  been  developed  with  a  collaborative  aspect 
(e.g.,  networked  computer  systems  where  teams  interact  with  the  world  together),  it  seems  as 
though  there  are  still  fundamental  aspects  of  collaboration  that  still  need  additional  research  if  they 
are  to  be  implemented,  or  would  be  very  challenging  to  implement  (e.g.,  the  ability  to  read  and 
interpret  body  language,  talk  face-to-face,  have  informal  gatherings  where  people  get  to  know  one 
another,  etc.)  However,  some  aspects  of  collaboration  and  related  challenges  could  probably  be 
implemented  in  microworlds.  For  example,  simulating  collaboration  within  a  distributed  team  and 
components  of  dealing  with  different  languages  (i.e.,  some  effects  of  culture  as  well  as 
collaboration)  and  terminology  have  likely  been  successfully  implemented  in  a  microworld  (but  a 
thorough  review  of  that  literature  was  beyond  the  scope  of  this  project). 

4.5  Additional  insights 

4.5.1  The  importance  of  social  factors 

The  importance  of  social  factors  in  decision  making  contexts  experienced  by  CF  personnel  cannot 
be  overemphasized.  Social  ties  and  social  networks  are  critical  for  the  way  that  CF  personnel 
typically  achieve  their  goals.  Indeed,  social  factors  were  usually  one  of  the  first  things  that  the 
SMEs  mentioned  during  their  interviews,  and  most  SMEs  mentioned  specific  education  and 
training  suggestions  related  to  enhancing  the  ability  to  build  social  relationships. 

One  major  difference  between  the  decision  making  experienced  by  CF  personnel  and  the  types  of 
decision  making  explored  in  the  complexity  literature  that  we  reviewed  is  that  real-life  decision 
makers  usually  have  to  act  in  the  context  of  other  people,  and  there  are  a  lot  of  ways  this  impacts 
the  situation.  Usually  goals  involve  the  state  of  other  people  (e.g.,  defeating  an  enemy),  and  usually 
the  actions  of  the  decision  maker  are  implemented  through  the  combined  actions  of  others  (e.g.,  in 
a  military  unit  the  soldiers  actually  carry  out  the  decisions  of  the  commander,  as  on  his  or  her  own 
the  commander  could  do  very  little  to  implement  his  or  her  decisions).  Also,  decision  making  is 
often  a  social  (i.e.,  team-based)  activity  rather  than  strictly  an  individual  activity. 

The  effective  representation  of  social  factors  in  microworlds  requires  more  research  (at  least  based 
on  the  literature  reviewed  for  this  project)  and  it  is  unknown  how  it  could  be  implemented 
effectively  within  the  paradigm  of  a  single  user  interacting  with  a  simulation,  typical  of  most 
microworld  research.  Not  only  is  the  social  context  fundamental,  but  it  creates  the  risk  of  different 
types  of  errors;  for  example,  errors  in  implementation  due  to  misunderstandings  etc.,  which  makes 
communication  critical.  Note  that  there  have  been  attempts  to  use  virtual  environments  to  teach 
people  to  be  more  effective  decision  makers  when  dealing  with  cultural  challenges;  these  are 
discussed  in  the  cultural  section  below. 

It  is  worth  considering  that  social  factors  may  be  fundamentally  different  from  other  aspects  of 
complex  decision  making  environments.  It  is  possible  that  people  process  information  related  to 
social  factors  differently  than  other  contextual  factors,  and  find  implications  related  to  social 
factors  easier  to  understand.  One  of  the  most  difficult  aspects  of  complex  decision  making  is 
generally  thought  to  be  appreciating  dynamic  aspects  of  the  environment;  for  example,  time-lagged 
effects  (i.e.,  when  causes  and  effects  are  separated  in  time)  are  often  not  fully  understood  by 
decision  makers  (e.g.,  Domer,  1996).  However,  one  of  the  main  time-lagged  effects  that  the  SMEs 
seemed  to  understand  very  well  and  mentioned  frequently  was  the  benefit  of  creating  positive 
social  relationships  and  social  networks,  and  also  the  cost  of  creating  negative  social  relationships. 
The  creation  and  maintenance  of  positive  social  relationships  was  mentioned  as  a  specific  goal  with 
immediate  benefits,  but  it  was  also  usually  mentioned  that  these  positive  social  connections  would 
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likely  pay  off  in  some  tangible  way  in  the  future.  For  example,  people  understand  that  when  they 
do  a  favour  for  someone,  this  is  generally  a  good  thing,  because  if  they  require  help  in  return 
someday  they  are  more  likely  to  get  it.  Thus,  some  aspects  of  social  relationships  etc.  are  seen  a 
setting  the  stage  for  future  benefits.  In  fact,  building  a  social  network  is  a  form  of  risk  management, 
putting  something  “in  the  bank”  as  a  way  of  solving  unknown  future  problems. 

It  is  possible  that  this  relatively  developed  ability  to  understand  complexity  as  related  to  social 
issues  may  have  contributed  to  the  fact  that  social  factors  have  typically  been  ignored  in  the 
complexity  literature.  Intuitively,  the  definition  of  what  is  “complex”  is  based  on  what  humans  are 
poor  at  doing.  That  is,  if  humans  can  do  something  relatively  easily,  then  it  is  not  seen  as  complex. 
However,  it  is  not  clear  that  social  factors  should  be  ignored  by  complexity  researchers  simply 
because  humans  for  the  most  part  are  better  at  understanding  the  implications  of  social  factors  than 
other  types  of  factors.  Indeed,  as  social  factors  appear  to  be  so  critical  to  CF  decision  making 
contexts,  it  would  seem  as  though  these  factors  deserve  a  great  deal  of  attention. 

4. 5.1.1  Culture 

Culture  appeared  as  a  general  challenge  theme,  and  it  is  closely  related  to  social  factors.  While 
culture  was  narrowly  defined  in  the  challenges  analysis,  cultural  issues  also  come  into  play  when 
different  organizations  or  other  groups  have  to  work  together  (these  types  of  challenges  were 
generally  included  in  the  Collaboration  challenge  theme).  The  current  focus  in  the  CF  on  Joint, 
Interagency,  Multinational,  and  Public  (JIMP)  operations  means  that  challenges  related  to  culture 
are  likely  to  increase,  and  education  and  training  programs  should  be  developed  to  prepare  CF 
personnel  to  operate  effectively  in  the  midst  of  such  challenges. 

An  overview  of  research  into  what  factors  contribute  to  the  ability  to  competently  deal  with  other 
cultures  (Adams  &  Brown,  2011)  provided  evidence  supporting  our  claim  that  social  factors  are 
likely  to  be  important  in  CF  decision  making.  Factors  that  Adams  and  Brown  found  to  be  important 
for  cultural  competency  which  overlap  with  the  social  factors  our  SMEs  thought  were  important 
included: 

•  Ability  to  negotiate 

•  Emotional  stability  and  self-regulation  (e.g..  Emotional  Quotient) 

•  Relationship  building 

•  Influence  and  persuasion 

•  Extraversion 

•  Agreeableness 

•  Conflict  resolution 

Thus,  independent  research  has  found  that  the  importance  of  social  factors  is  likely  to  play  a  large 
role  in  the  success  of  CF  operations,  and  CF  personnel  should  be  adequately  supported  in  the 
development  of  these  skills  and  attributes. 

Adams  and  Brown  (2011)  also  noted  that  there  are  immersive  environments  and  virtual  humans 
being  used  in  cultural  competence  instruction.  Immersive  environments  can  help  to  provide  a 
visual  representation  of  another  environment  (e.g.,  dress,  architecture)  and  get  personnel  used  to 
situations  that  they  will  encounter  in  the  future.  For  example,  the  Tactical  Iraqi  Language  and 
Culture  Training  System  (TLCTS)  allows  users  to  explore  a  virtual  Iraqi  village,  and  includes 
language,  local  sounds,  buildings,  etc.  which  allow  a  sense  of  a  real  Iraqi  village.  The  Adaptive 
Thinking  and  Leadership  (ATL)  is  a  team-training  system  in  which  some  users  experience  both 
sides  of  intercultural  scenarios.  Users  take  turns  playing  people  from  different  cultures,  and  are 
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provided  with  backstories  and  goals  to  make  their  experience  more  accurate.  Both  of  these  systems 
can  be  adjusted  to  give  feedback  about  cultural  errors. 

Although  it  is  possible  that  systems  similar  to  the  TLCTS  and  the  ATL  could  be  modified  to  create 
microworlds  to  help  train  cultural  competency,  and  perhaps  even  additional  social  factors,  Adams 
and  Brown  (2011)  noted  that  explicit  feedback  from  a  human  instructor  is  required  to  facilitate 
learning.  As  well,  these  sorts  of  training  systems  teach  cultural  specific  information  and  do  little  to 
teach  the  factors  mentioned  earlier  in  this  section  as  generally  required  for  effective  decision 
making  (e.g.,  ability  to  negotiate,  emotional  stability,  etc.)  Thus,  it  is  likely  that  virtual 
environments  and  microworlds  will  continue  to  be  inadequate  for  teaching  cultural  issues  in 
complex  decision  making  effectively. 

4.5.2  Poor  and  good  actor  behaviours 

As  noted  previously,  originally  it  was  hoped  that  we  could  gather  information  about  good  and  poor 
actor  behaviours  related  to  complex  decision  making,  and  determine  which  of  these  should  be 
taught  so  that  CF  personnel  would  be  better  able  to  handle  complex  decision  making  situations. 
Unfortunately,  it  was  difficult  to  accomplish  this,  for  several  reasons.  As  already  discussed,  many 
of  the  good  and  poor  actor  behaviours  are  very  difficult  to  empirically  assess.  Another  reason  that  it 
proved  to  be  very  difficult  to  examine  good  or  poor  actor  decision  making  behaviours  was  because 
it  was  unknown  how  well  the  SMEs  understood  their  own  decision  making  behaviours  and 
environments.  In  some  cases,  SMEs  had  difficulty  describing  their  decisions  and  decision  making 
processes,  which  made  it  difficult  to  assess  their  behaviours  as  good  or  bad.  Further,  there  were 
very  few  examples  provided  of  bad  actor  behaviours  on  the  part  of  the  DM  themselves.  It  is 
unknown  whether  this  was  due  to  report  bias  on  the  part  of  the  SMEs,  whether  it  was  because  they 
really  didn’t  perform  any  poor  actor  behaviours,  or  whether  there  was  a  lack  of  understanding  of 
what  would  constitute  good  or  poor  actor  behaviour. 

It  is  worth  noting  that  there  is  some  evidence  that  attempting  to  find  behaviours  that  are  always 
“good”  or  “poor”  may  be  misguided,  and  that  the  critical  ability  of  a  DM  in  a  complex  environment 
is  knowing  what  behaviour  to  display  when.  That  discussion  is  the  topic  of  the  next  section. 

4.5.3  Knowing  when  to  do  what 

Complex  Adaptive  Systems  (CAS)  (e.g.,  Grisogono,  2006)  have  often  been  studied  to  understand 
aspects  of  complexity.  CAS  are  so  named  because  they: 

1.  Are  a  set  of  interacting  or  interdependent  components  (i.e.,  a  system), 

2.  Are  composed  of  many  parts  which  as  a  whole  exhibit  behaviours  not  predictable  from 
individual  components  (i.e.,  they  are  complex),  and 

3.  Show  individual  and  collective  behaviour  change  as  a  result  of  experience  (i.e.,  they 
adapt). 

Examples  of  CAS  include  the  stock  market,  social  insect  colonies,  the  brain,  and  the  immune 
system. 

The  study  of  CAS  makes  apparent  that  effectively  adapting  to  a  complex  environment  may  require 
showing  different  properties  at  different  times.  For  example,  Grisogono  (2006)  discusses  the 
“paradoxical”  nature  of  CAS  by  pointing  out  the  fact  that  many  of  the  properties  of  CAS  appear  to 
conflict  with  one  another.  An  example  given  is  that  “robustness  to  damage”  requires  that  some 
changes  in  the  CAS  are  inhibited,  while  “innovation”  requires  that  some  changes  in  the  CAS  are 
amplified.  Thus,  it  may  not  be  possible  to  determine  certain  properties  which  are  always  adaptive; 
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rather,  what  makes  a  CAS  (or  DM)  effective  might  be  the  combination  of  having  an  adequate  range 
of  behaviours  for  the  situations  it  encounters,  as  well  as  the  ability  to  appropriately  shift  between 
this  range  of  behaviours  at  the  proper  times  for  optimal  results. 

The  fact  that  contrasting  behaviours  may  be  required  to  create  an  effective  CAS  has  serious 
implications  for  teaching  CF  personnel  to  be  better  decision  makers.  It  might  not  be  so  important  to 
teach  the  DM  to  use  particular  skills,  but  rather  the  DM  needs  to  be  able  to  do  a  large  number  of 
different  things,  and  the  most  important  thing  is  knowing  WHEN  to  do  WHAT. 

4.6  Limitations 

There  are  several  limitations  related  to  the  research  presented  in  this  report.  Some  limitations  are 
related  to  the  data  collection  method,  some  are  related  to  the  data  analysis  method,  some  are  related 
to  the  scope  of  the  report,  and  some  are  due  to  practical  limitations  related  to  recruitment  of  SMEs. 

Some  limitations  have  to  do  with  the  way  that  data  was  collected  for  this  report.  Although  we 
acquired  a  lot  of  useful  information,  because  the  data  we  collected  was  elicited  via  interviews,  it  is 
possible  that  a  lot  of  information  was  missed.  It  is  possible  that  the  interview  questions  were  not 
adequate  to  elicit  all  of  the  important  information  from  all  SMEs.  This  could  have  been  because 
SMEs  might  not  have  really  appreciated  all  aspects  of  their  complex  decision  making  situations.  As 
well,  in  some  cases  years  had  elapsed  between  the  events  we  were  asking  about  and  the  time  of  the 
interview.  Thus,  there  could  be  missing  information  because  SMEs  did  not  appreciate  the 
importance  of  it  at  the  time,  have  since  forgotten,  or  memory  may  be  biased  (e.g.,  availability 
heuristic,  hindsight  bias;  see  Adams,  Rehak,  Brown,  &  Hall,  2009  for  a  review  of  decision-making 
biases).  In  addition,  two  hours  was  not  sufficient  time  to  gather  all  relevant  information  about 
many  of  the  scenarios.  It  is  reasonable  that  more  time  would  have  been  required  to  fully  explore  all 
aspects  of  a  complex  decision  making  domain,  and  only  the  highest  level  aspects  may  have  been 
assessed  in  this  study.  It  is  also  possible  that  information  was  not  gathered  because  SMEs  could  not 
articulate  aspects  that  they  felt  were  important  in  their  decision  making  (i.e.,  it  may  be  difficult  to 
describe  complex  decision  making  in  words). 

The  education  and  training  which  SMEs  have  received  might  also  have  influenced  the  data  we 
could  collect  for  this  project.  In  addition  to  shaping  their  behaviour,  it  is  highly  likely  that  SMEs’ 
education  and  training  influences  their  view  of  situations  they  encounter  and  shapes  their 
understanding  of  the  situation.  This  in  turn  would  influence  the  information  they  would  provide 
during  interviews  about  these  situations.  For  example,  if  SMEs  are  taught  to  pay  attention  to  certain 
aspects  of  their  decision  making  environment  (e.g.,  who  the  authority  figures  are),  then  these 
aspects  might  prove  to  be  more  salient  to  the  SMEs  than  other  aspects,  regardless  of  the  amount 
each  decision  making  aspect  actually  impacted  the  SMEs’  decisions. 

Because  we  were  asking  individuals  about  events  they  had  experienced  and  we  had  little  empirical 
evidence  to  compare  the  SMEs’  statements  with,  it  was  unknown  the  extent  to  which  any  bad 
outcomes  were  the  result  of  poor  decision  making  behaviours  or  whether  the  situation  was  just 
extremely  difficult.  For  example,  several  SMEs  mentioned  that  they  thought  that  they  basically 
tried  everything  and  nothing  could  be  done  to  improve  the  situation.  It  is  unclear  whether  this  was 
the  case  (i.e.,  there  was  a  set  point  in  the  environment  that  was  highly  resistant  to  change),  or 
whether  lack  of  change  was  the  result  of  poor  decision  making. 

Another  limitation  related  to  data  collection  was  that  there  were  some  sensitive  topics  which  SMEs 
did  not  wish  to  discuss,  or  which  they  did  not  wish  to  be  included  in  our  report.  This  limited  our 
understanding  and  representation  of  the  scenarios,  as  some  factors  could  not  be  included. 


Humwsystems "  Incorporated 


Page  57 


HUMAN.Vrsrf  MS 


There  were  also  limitations  related  to  the  analysis  method  used  in  this  project.  The  creation  of  the 
scenarios,  the  mapping  between  the  scenarios  and  the  complexity  concepts,  and  the  bottom-up 
extraction  of  challenges  and  education  and  training  suggestions  were  done  in  a  somewhat 
subjective  manner.  There  may  have  been  unknown  bias  on  the  part  of  the  analyst  which  might  have 
shaped  the  findings.  Future  work  could  use  more  objective  methods  for  data  analysis  (e.g.,  a  factor 
analysis). 

It  was  also  a  limitation  that  validating  the  findings  of  the  complexity  research  was  out  of  scope  for 
this  project,  and  the  amount  of  literature  reviewed  was  quite  small.  Thus,  it  is  possible  that 
important  factors  related  to  complexity  were  either  not  found  in  the  literature  search  (although  the 
literature  search  findings  were  validated  at  least  somewhat  in  the  workshop)  or  that  important 
factors  are  missing  from  the  complexity  literature.  In  some  cases  these  omissions  were  noted  (e.g., 
in  the  gap  analysis),  but  there  may  be  additional  missing  factors. 

We  were  limited  as  to  the  experiences  of  the  SMEs  who  were  recruited  for  this  project.  Some 
limitations  were  known  in  advance  (e.g.,  this  project  had  an  army  focus  rather  than  an  air  force  or 
navy  focus,  so  the  results  may  not  be  as  relevant  for  the  air  force  or  navy),  whereas  some  were  the 
result  of  recruitment  limitations  (e.g.,  humanitarian  expeditionary  operations  were  intended  to  be 
included  but  no  SMEs  were  recruited;  only  one  example  of  a  domestic  operation  was  included). 
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5.  Conclusions  and  Recommendations 


Challenging  decision  making  environments  such  as  those  experienced  by  the  CF  are  commonly 
being  characterized  as  “complex”  by  researchers  (e.g.,  Grisogono,  2010).  Flowever,  many  of  these 
claims  have  been  made  based  on  intuition,  rather  than  a  detailed  examination  of  whether  the  factors 
that  make  decision  making  “complex”  according  to  the  literature  actually  are  present  in  the 
environments  encountered  by  the  CF.  The  main  goal  of  this  project  was  to  determine  whether 
research  investigating  complex  decision  making  is  relevant  to  the  decision  making  actually 
experienced  by  Canadian  Forces  personnel,  and  how  that  research  might  be  used  to  improve 
Canadian  Forces  instruction  related  to  decision  making. 

A  literature  review  identified  major  components  of  what  makes  decision  making  difficult  according 
to  the  complexity  literature.  Five  factors  were  chosen  for  further  examination:  Connectivity, 
dynamics,  multiple  conflicting  goals,  underspecified  goals,  and  independent  agents.  Without 
exception,  we  found  that  these  five  factors  appeared  in  each  of  the  scenarios  describing  the 
experiences  of  the  SMEs  we  interviewed.  As  well,  the  SMEs  indicated  that  these  factors  do  indeed 
make  their  decision  making  more  difficult.  Thus,  it  appears  as  though  factors  identified  in  the 
complexity  literature  are  indeed  part  of  the  experience  of  CF  personnel. 

There  appear  to  be  some  types  of  scenarios  which  are  more  challenging  than  others.  For  example, 
operations  (both  domestic  and  expeditionary)  tend  to  contain  more  complexity  components  than 
domestic  day-to-day  functions.  It  also  appears  as  though  scenarios  tend  to  be  complex  on  all  five 
complexity  factors.  This  indicates  that  CF  personnel  who  are  going  to  engage  in  operations  likely 
require  more  support  to  improve  their  decision  making  abilities  than  personnel  who  are  principally 
engaging  in  domestic  day-to-day  functions. 

Although  there  was  significant  overlap  between  the  factors  identified  in  the  complexity  literature 
and  the  factors  which  appeared  to  make  decision  making  difficult  for  CF  personnel,  there  were  also 
significant  gaps  identified,  with  many  additional  challenges  facing  CF  personnel  that  were  not 
identified  in  the  complexity  literature  review.  The  most  important  of  these  appear  to  be  factors 
related  to  collaboration,  including  cultural  factors.  Other  areas  which  challenge  CF  personnel 
which  appear  to  require  additional  instruction  include  planning  and  dealing  with  resource 
challenges.  CF  education  and  training  could  also  be  enhanced  if  future  work  examines  which  of  the 
complexity  challenges  examined  in  this  project  are  the  most  challenging  for  CF  personnel,  and  aim 
interventions  at  increasing  DM’s  proficiency  in  those  areas. 

5.1  Future  Work 

Potential  future  work  which  could  be  logical  extensions  of  this  report  include  exploring  different 
data  collection  techniques,  using  different  data  analysis  techniques,  and  investigating  different 
complexity  and  education  and  training  topics. 

5.1.1  Data  collection 

Using  different  approaches  for  collecting  data  could  add  to  our  knowledge  about  how  complex 
decision  making  occurs  in  the  CF.  Conducting  interviews  with  a  two-hour  time  limit  allowed  us  to 
gather  useful  information  about  decision  making  in  complex  CF  environments  at  a  high  level,  but 
there  was  probably  a  lot  of  lower- level  information  that  was  not  collected.  For  example,  it  might  be 
useful  to  collect  more  information  about  how  SMEs  attempted  to  understand  the  factors  that  were 
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important  to  their  decision  making  as  this  could  be  used  to  determine  whether  there  are  general 
strategies  for  understanding  a  complex  situation.  Conducting  longer  interviews,  or  multiple 
interviews  with  the  same  SME,  could  allow  for  more  information  to  be  gathered. 

Now  that  some  information  has  been  gathered  about  the  types  of  challenges  which  play  a  role  in 
CF  complex  decision  making,  it  is  possible  that  data  collection  methods  could  incoiporate 
questionnaires  to  gather  more  data  about  aspects  of  these  challenges  such  as  their  frequency  of 
occurrence,  the  difficulty  of  dealing  with  the  challenges,  the  degree  of  impact  they  have  on  the 
outcome  in  different  situations,  etc. 

Data  collection  could  also  be  done  through  other  methods  such  as  observation.  One  way  of 
incorporating  observation  would  be  to  combine  observation  with  interviews  similar  to  the  method 
used  in  this  project,  if  the  scenarios  being  investigated  were  ongoing.  Observing  the  situation 
would  provide  a  more  complete  understanding  of  the  complexity  phenomena  that  could  be 
compiled  and  used  for  instructional  purposes.  Another  potentially  useful  way  to  collect  data  using 
observation  is  using  experienced  SMEs  to  observe  the  behaviour  of  novices  being  taught,  and  to 
get  the  SMEs  to  comment  on  things  that  the  novices  are  doing  well  and  what  they  are  doing  poorly. 
If  there  is  some  objective  way  to  determine  what  the  novices  are  actually  doing  (e.g.,  they  are  being 
instructed  using  microworlds  which  record  their  actions  and  the  outcomes)  this  information  could 
prove  valuable  for  understanding  what  components  of  decision  making  the  SMEs  are  actually 
considering  when  determining  whether  the  novices  are  performing  well  or  not.  This  would  be 
particularly  beneficial  when  considering  that  experts  often  do  have  good  insight  into  what  novices 
are  doing  well  or  doing  poorly,  but  may  have  difficulty  articulating  exactly  what  is  good  or  poor 
about  the  novices’  behaviours.  As  well,  experts  often  do  not  have  a  conscious  understanding  of 
how  they  determine  that  a  novice’s  behaviours  are  good  or  poor,  or  their  conscious  understanding 
does  not  actually  reflect  the  information  that  they  are  actually  considering.  Using  an  observational 
method  such  as  the  one  described  here  would  overcome  many  of  these  limitations. 

5.1.2  Data  analysis 

It  would  also  be  possible  to  analyse  data  via  a  factor  analysis.  The  benefit  of  performing  a  factor 
analysis  of  complexity  would  be  to  determine  which  factors  influence  the  perception  of  complexity 
from  a  CF  perspective.  This  could  be  used  to  guide  our  understanding  of  how  CF  personnel 
perceive  complexity  and  enable  education  and  training  enhancements  to  be  presented  in  terms 
intuitive  to  those  being  taught.  To  perform  a  factor  analysis  of  complexity,  researchers  could  ask 
SMEs  to  rate  the  similarity  of  the  complexity  of  pairs  of  scenarios  or  situations.  These  ratings 
would  be  analysed  to  determine  the  underlying  mental  structure  of  complexity  (i.e.,  what  factors 
appear  to  determine  what  makes  a  situation  complex  in  the  SMEs’  view).  One  of  the  benefits  of 
this  approach  is  that  it  is  a  data-driven  (i.e.,  bottom- up)  approach,  so  the  SMEs  would  be  telling  us 
what  complexity  is  rather  than  us  trying  to  fit  their  statements  into  our  structure.  However,  it  would 
not  necessarily  tell  us  about  their  performance. 

The  data  analysis  methods  used  in  this  project  were  largely  subjective.  Semantic  analysis  is 
sometimes  used  to  determine  the  main  ideas  and  their  relationships  within  written  documents,  and 
such  an  analysis  would  provide  a  more  objective  way  of  categorizing  the  experiences  of  CF  SMEs. 

Finally,  if  questionnaires  are  used  to  collect  data,  it  would  be  possible  to  perform  correlations 
between  factors  which  have  been  rated  in  questionnaires.  For  example,  if  SMEs  are  used  to  rate  the 
performance  of  novices,  and  information  is  gathered  about  what  those  novices  are  actually  doing, 
those  two  sets  of  data  can  be  correlated  to  determine  which  behaviours  actually  tend  to  result  in 
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higher  rankings  from  the  SMEs,  and  experimenters  will  not  have  to  rely  on  the  ability  of  SMEs  to 
articulate  what  they  think  is  important  in  complex  decision  making. 

5.1.3  Additional  topics 

Determining  whether  there  are  distinct  types  of  behaviours  which  are  generally  effective  or 
ineffective  for  complex  decision  making  have  been  of  interest  to  researchers  (e.g.,  Grisogono, 
2010).  In  this  project,  it  proved  to  be  extremely  difficult  to  assess  whether  DMs  showed  good  or 
poor  actor  behaviours,  and  this  was  partially  due  to  behaviour  descriptions  being  vague  and  not 
easily  measured.  Future  work  could  be  done  to  refine  the  good  and  poor  actor  behaviour 
descriptions  and  develop  a  method  to  empirically  assess  the  presence  of  those  good  and  poor  actor 
behaviours.  This  could  in  turn  facilitate  the  assessment  of  the  viability  of  particular  decision 
making  behaviours,  and  determine  whether  there  are  general  behaviours  that  should  be  taught  to 
facilitate  CF  decision  making.  Note  that  this  work  should  take  into  consideration  that  the  factors  of 
importance  may  be  “meta-cognitive”,  or  related  to  understanding  what  behaviours  to  produce 
when,  than  related  to  specific  types  of  behaviours  (e.g.,  always  making  goals  concrete). 

There  was  evidence  that  the  five  main  complexity  factors  (connectivity,  dynamics,  multiple 
conflicting  goals,  underspecified  goals,  and  independent  agents)  were  all  present  to  some  extent  in 
the  10  scenarios  created  for  this  project.  However,  because  the  five  main  complexity  factors  were 
only  compared  within  the  set  of  10  scenarios  produced  for  this  project,  it  is  difficult  to  make 
definitive  conclusions  about  the  relative  difficulty  of  the  different  complexity  factors  in  CF 
decision  making.  Future  work  could  be  performed  to  address  this  issue.  Possible  methods  include 
attempting  to  create  a  general  assessment  measure  for  the  complexity  factors,  perhaps  with  a 
different  means  of  collecting  data  (e.g.,  observation  of  exercises). 

It  could  be  useful  to  investigate  differences  between  novice  and  expert  decision  makers  in  complex 
CF  domains.  Assuming  that  there  are  differences,  understanding  these  differences  could  help  to 
streamline  learning,  if  the  different  behaviours  that  experts  engage  in  tend  to  result  in  better 
decision  making.  As  well,  novices,  although  they  are  not  generally  as  effective  as  experts,  often 
have  better  insight  into  how  they  are  making  decisions,  and  it  would  be  useful  to  understand  what 
novices  find  difficult.  It  is  possible  that  novices  find  different  decision  making  components  more 
difficult  than  experts  do,  and  experts  were  the  focus  of  this  project.  It  may  be  that  instruction  in 
complex  decision  making  should  occur  in  stages,  with  more  basic  skills  having  to  be  acquired 
before  advanced  strategies  for  dealing  with  complex  decisions  can  be  effectively  taught  or  applied 
by  the  decision  maker. 

It  should  be  noted  that,  if  no  consistent  differences  are  found  between  novice  and  expert  decision 
makers,  this  might  indicate  that  instruction  should  focus  on  general  rather  than  context  specific 
strategies  for  making  complex  decisions.  That  is,  if  novice  and  expert  decision  makers  do  not 
appear  to  differ  in  consistent  ways,  it  might  indicate  that  there  is  no  situation  specific  information 
or  experience  that  benefits  the  decision  maker.  This  is  another  area  of  research  which  should  be 
further  investigated.  If  there  are  strategies  for  making  complex  decisions  that  can  be  widely  and 
effectively  applied  across  a  variety  of  situations,  it  would  be  of  extreme  benefit  to  teach  these 
strategies  rather  than  focus  instruction  on  situation  specific  strategies.  However,  if  there  are 
situation  specific  elements  that  should  be  taught  to  support  effective  decision  making  in  complex 
CF  domains,  this  should  be  known  and  these  strategies  need  to  be  described. 

Another  area  for  future  research  would  be  investigating  possible  interactions  between  effective 
decision  making  strategies  and  individual  differences.  It  is  very  possible  that  there  are  general 
strategies  that  are  effective  in  multiple  decision  making  domains,  but  that  different  people  will  have 
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different  levels  of  success  at  applying  these  strategies.  For  example,  it  would  be  possible  that 
someone  with  a  large  working  memory  (i.e.,  they  can  keep  track  of  more  pieces  of  information  and 
understand  connections  between  them  than  average)  might  be  able  to  keep  track  of  many  different 
factors  at  the  same  time  and  consider  how  they  might  interact  with  one  another.  Such  an  individual 
could  benefit  from  instruction  about  which  factors  tend  to  be  important  and  how  they  tend  to 
interact  in  a  specific  domain.  However,  if  that  person  is  very  shy,  they  might  not  benefit  from 
learning  which  individuals  were  important  in  a  particular  context,  because  they  might  find  it  very 
difficult  to  create  positive  social  interactions  with  these  individuals.  On  the  other  hand,  someone 
who  was  very  extraverted  but  with  a  smaller  working  memory  size  might  show  the  opposite  pattern 
and  benefit  much  more  from  learning  with  whom  it  might  be  beneficial  to  create  positive  social 
relationships,  but  benefit  less  from  learning  all  of  the  factors  which  might  impact  their  decisions  as 
they  would  have  less  ability  to  reason  about  the  interrelationships  between  them. 


Page  62 


Humanly, stem, s1'  Incorporated 


HUMAN  SYSTEMS 

O’Jttd 


References 


Adams,  B.  D.  &  Brown,  A.  L.  (2011).  Competencies  within  the  public  domain:  Literature  review 
(DRAFT).  Toronto,  ON:  Defence  Research  and  Development  Canada. 

Adams,  B.  D.,  Rehak,  L.,  Brown,  A.,  &  Flail,  C.  D.  T.  (2009).  Fluman  decision-making  biases. 
(DRAFT).  Toronto,  ON:  Defence  Research  and  Development  Canada. 

Artman,  H.  (1999).  Situation  awareness  and  co-operation  within  and  between  hierarchical  units  in 
dynamic  decisionmaking.  Ergonomics,  1999,  42,  1404-1417.Brehmer,  B.  &  Allard,  R.  (1991). 
Dynamic  decision-making:  The  effects  of  task  complexity  and  feedback  delay.  In  J.  Rasmussen,  B. 
Brehmer,  &  J.  Leplat  (Eds.),  Distributed  Decision-Making:  Cognitive  Models  for  Cooperative 
Work  (pp.  319-334).  Oxford,  England:  John  Wiley  &  Sons. 

Brehmer,  B.  &  Domer,  D.  (1993).  Experiments  with  computer-simulated  microworlds:  Escaping 
both  the  narrow  straits  of  the  laboratory  and  the  deep  blue  sea  of  the  field  study.  Computers  in 
Fluman  Behavior,  9,  171-184. 

Brown,  A.,  Karthaus,  C.,  Rehak,  L.,  &  Adams,  B.  (2009).  The  role  of  mental  models  in  dynamic 
decision-making.  DRDC  Report  No.  CR-2009-06Q.  Toronto,  ON:  Defence  Research  and 
Development  Canada. 

Clancy,  J.  M.,  Elliott,  G.  C.,  Ley,  T.,  Omodei,  M.  M.,  Wearing,  A.  J.,  McLennan,  J.,  & 
Thorsteinsoon,  E.  B.  (2003).  Command  style  and  team  performance  in  dynamic  decision-making 
tasks.  In  S.  Schneider  &  J.  Shanteau  (Eds.),  Emerging  Perspectives  in  Decision  Research  (pp.  586- 
619).  Cambridge  University  Press. 

Department  of  Defence  (2009).  Adaptive  campaigning  09:  Army’s  future  land  operating  concept, 
Australian  Army,  Canberra. 

Dorner,  D.  (1996).  The  Logic  of  Failure:  Why  Things  Go  Wrong  and  What  We  Can  do  to  Make 
them  Right.  New  York:  Metropolitan  Books. 

Funke,  J.  (2001).  Dynamic  systems  as  tools  for  analysing  human  judgement.  Thinking  and 
Reasoning,  7,  69-89. 

Godefroy,  A.B.  (Ed.).  (2007).  Land  Operations  2021:  Adaptive  Dispersed  Operations.  Ed: 
Directorate  of  Land  Concepts  and  Design,  Government  of  Canada,  D2-188/2007E 

Grisogono,  A.  M.  (2010).  Overview  of  Complex  Decision-Making  (CxDM)  program  (DRAFT). 

Grisogono,  A.  M.  (2006).  The  implications  of  complex  adaptive  systems  theory  for  C2.  Paper 
presented  at  CCRTS:  The  State  of  the  Art  and  the  State  of  the  Practice. 

Jarmasz,  J.,  Rehak,  L.,  Taylor,  T.,  Bruyn  Martin,  L.,  Karthaus,  C.,  and  Vokac,  R.,  (2011).  Survey 
of  Dynamic  and  Complex  Decision  Making  Experiences  in  Canadian  Forces  Land  Operations. 
DRDC  Ethics  Protocol  #L-762.  Toronto,  ON:  Defence  Research  and  Development  Canada. 

McLennan,  J.,  Omodei,  M.,  Flolgate,  A.,  &  Wearing,  A.  J.  (2003).  Fluman  information  processing 
aspects  of  effective  emergency  incident  management  decision  making.  Paper  presented  at  the 
Fluman  Factors  of  Decision  Making  in  Complex  Systems  Conference,  Dunblane,  Scotland. 

Moffat,  J.  (2003).  Complexity  theory  and  network  centric  warfare.  CCRP  Publication  Series, 
available  for  download  at  www.dodcci-p.org. 

Human  systems !"  Incorporated 


Page  63 


HUMAN.Vrsr£WS 


Rehak,  L.,  Lamoureux,  T.,  &  Bos,  J.  (2006).  Systems  archetypes  for  military  dynamic  decision 
making.  DRDC  Report  No.  CR-2006-202.  Toronto,  ON:  Defence  Research  and  Development 
Canada. 

Rehak,  L.  A.,  Taylor,  T.  E.,  Karthaus,  C.,  &  Bruyn  Martin,  L.  (2010).  Cue  integration  in  dynamic 
decision  making.  DRDC  Report  No.  CR-20 10-047.  Toronto,  ON:  Defence  Research  and 
Development  Canada. 

Thomson,  M.H.,  Adams,  B.D.,  Hall,  C.D.,  Brown,  A.L.,  Flear,  C.  (2011).  Interagency  Trust  in  the 
Whole  of  Government  Approach  to  Canadian  Forces  Operations  (DRAFT).  Toronto,  ON:  Defence 
Research  and  Development  Canada. 


Page  64 


Y\umwsystems  Incorporated 


Annex  A:  Mind  Map  of  Complexity  Factors 
and  Behaviours 

See  the  Mind  Map. 
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Annex  B:  Operational  Scenarios  that  Contain 
Complexity 


This  annex  contains  the  operational  scenario  descriptions.  The  scenarios  are  presented  in  order  of 
overall  complexity  ranking  (see  Table  10),  with  the  most  complex  scenario  first. 

1.  Scenario  Description:  Military  Liaison  and  Advisor,  International 
Event 

The  decision  maker  (DM)  was  assigned  to  be  the  military  liaison  and  advisor  during  a  major 
international  event.  It  was  the  responsibility  of  the  DM  to  facilitate  communications  between  the 
stakeholders  (including  the  CF  and  other  municipal,  provincial,  and  federal  authorities)  and  keep 
the  DM’s  superior  officers  informed.  The  DM  had  to  decide  what  information  to  communicate, 
with  whom  to  communicate,  and  when  to  communicate  it.  During  the  event  the  DM  was  present  in 
the  Local  Command  Centre  (LCC)  and  had  to  “manage”  the  sensitive  time  when  a  breakdown  in 
C2  occurred  during  which  security  forces  were  ordered  to  “stand  down”  and  the  security  situation 
surrounding  the  event  was  in  jeopardy. 

Factors  which  influenced  the  ability  of  the  DM  to  manage  these  processes  included: 

•  The  international  event  was  held  at  the  same  time  as  another  important  international  event. 
These  events  had  never  taken  place  simultaneously  before,  meaning  that  there  was  a  lack 
of  previous  events  to  draw  on  for  lessons  learned  etc.; 

•  In  general  there  was  a  high  demand  for  information  flowing  in  many  directions  at  once 
which  was  difficult  to  manage; 

•  Security  was  not  the  only  consideration  in  choosing  the  location  for  the  event,  therefore  the 
event  took  place  at  a  less  than  ideal  location  for  security  planning; 

•  Other  security  issues  included: 

o  There  was  an  extremely  large  local  population; 

o  There  were  many  public  order  units  involved,  creating  great  coordination 
problems; 

o  Water  access  (increased  accessibility); 

o  Close  to  an  international  border; 

•  Morale  among  some  personnel  suffered  as  some  security  stakeholders  resented  their 
considerations  not  carrying  the  same  weight  as  other  factors;  The  DM  had  extensive 
experience  as  a  police  officer  as  well  as  a  military  officer.  In  many  cases  this  worked  to  the 
DM’s  benefit  (e.g.,  DM  understood  police  procedure  and  had  more  credibility  with  the 
police)  but  sometimes  was  a  disadvantage  (e.g.,  sometimes  the  CF  people  would  feel  that 
DM  might  be  more  loyal  to  the  police  than  the  army); 

•  The  DM  replaced  another  individual  of  higher  rank;  because  the  DM  was  of  lower  rank, 
this  led  to  the  perception  by  the  police  that  the  CF  saw  this  position  as  being  less 
important; 
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•  The  DM  knew  the  personalities  of  several  people  involved  and  could  predict  how  they 
could  lead  to  problems  (e.g.,  knew  some  of  the  higher-level  officers  had  more  self-interest 
in  mind). 

•  Collaboration  between  the  federal  police,  the  local  police,  and  the  CF  was  made  more 
difficult  because  of  different  jargon  (e.g.,  OPP  to  the  CF  means  Operational  Planning 
Process  but  to  the  police  it  is  Ontario  Provincial  Police); 

•  Issues  that  arose  were  often  sensitive,  both  legally  and  politically  -  highly  political 
situation; 

•  There  was  massive  media  coverage  that  was  not  necessarily  favourable  to  the  international 
event; 

•  Security  measures  caused  a  number  of  unintended  effects  for  local  residents.  Freedom  of 
movement  was  restricted  and  security  measures  caused  unwanted  psychological  effects 
(e.g.,  anxiety,  isolation); 

•  Logistics  planning  was  not  de-conflicted  with  the  establishment  of  security  measures, 
which  meant  that  moving  supplies  (e.g.,  getting  food  to  personnel)  was  difficult.  There  was 
a  short  timeline  for  planning  changes  as  the  decision  to  restrict  vehicle  traffic  inside  the 
inner  security  cordon  was  not  shared  with  stakeholders  until  less  than  2  weeks  before  the 
event;  There  were  misunderstandings  about  what  the  CF  were  willing  and  able  to  provide: 

o  CF  personnel  were  not  properly  equipped  to  be  augment  police  or  security  forces 
should  the  need  arise  (they  only  had  lethal  force  options  available,  they  did  not 
have  proper  personal  protective  equipment); 

o  Inappropriate  requests  for  unskilled  assistance  were  made  (e.g.,  a  request  was 
made  for  CF  personnel  to  deliver  meals); 

o  There  was  a  huge  disconnect  between  what  local  security  forces  were  requesting 
and  what  federal  authorities  were  willing  to  provide  in  terms  of  financing; 

o  Not  all  CF  assets  were  declared  to  the  security  partners.  Some  security  partners  felt 
that  all  CF  assets  should  have  been  made  available; 

o  Not  all  CF  assets  were  declared  to  the  security  partners  although  some  were 
obviously  staged  forward  and  in  plain  sight; 

o  There  were  requests  for  assistance  that  were  incomplete  and  the  DM  had  to  go  to  a 
lot  of  work  to  determine  the  details  of  needs.  For  example,  a  request  from  a 
security  partner  to  use  DND  property  started  as  a  need  for  a  small  external  comer 
of  land  evolved  into  a  request  for  the  use  of  an  entire  CF  armoury  and  supplies; 

o  The  CF  were  frequently  requested  to  conduct  activities  outside  its  lawful  mandate, 
which  was  frequently  misunderstood  by  other  security  partners.  Inventive  solutions 
were  found  to  allow  the  CF  to  support  other  security  partners  to  conduct 
surveillance  as  required; 

•  Everything  was  being  recorded  in  the  cells  during  the  international  event,  so  this  led  to 
people  not  wanting  to  talk  in  case  they  said  something  that  could  be  used  against  them 
later; 
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•  What  was  happening  locally,  funding  issues,  etc.  meant  that  police  forces  might  or  might 
not  send  security  forces  to  support  the  event  (this  meant  that  the  number  of  security  forces 
to  be  present  was  not  certain  until  the  last  minute); 

•  Civilian  police  security  partners  conducted  independent  intelligence  gathering  despite  the 
agreement  that  the  operation  would  be  joint  and  interagency; 

•  When  new  costs  for  security  were  published  about  a  month  before  the  planned  event  this 
caused  a  reassessment  of  what  was  needed  (“we  can’t  pay  for  it  so  we’re  not  going  to  do 
it”),  everyone  was  worried  about  the  municipal  budget; 

•  The  DM  was  surprised  at  the  lack  of  information  coming  out  to  the  public  about  what  was 
going  on  (e.g.,  where  security  boundary  zones  would  go); 

•  Some  tactical  judgements  during  the  crowd  confrontation  operations  were  unsound.  Not 
all  personnel  were  donning  their  Individual  Protective  Equipment  when  certain  techniques 
were  employed,  leaving  police  personnel  vulnerable; 

•  Everyone  did  not  want  to  criticize  others  or  hear  negativity  and  so  plans  and  situations 
were  not  objectively  assessed; 

•  Because  this  was  such  a  large  and  important  event,  careers  were  on  the  line  which 
motivated  people  to  be  self-protective; 

•  General  Rules  of  Engagement  for  CF  acting  in  this  situation  were  at  times  obscure, 
depending  on  location,  (e.g.,  if  they  had  to  defend  their  assets); 

•  Asa  liaison,  the  DM  had  no  authority  to  give  direction  (understandably,  DM  could  only 
give  advice,  provide  information,  and  put  people  in  contact  with  one  another).  The  DM  did 
not  have  the  authority  to  intercede  in  policing  issues  and  there  were  cases  where  DM 
thought  changes  were  needed  (e.g.,  DM  was  surprised  at  the  lack  of  operational-level 
planning). 

Factors  which  unfolded  over  time: 

•  Strategic  level  decisions  were  made  at  the  latest  possible  moment.  This  meant  that  the  rate 
at  which  plans  had  to  be  made  and  implemented  kept  changing,  and  eventually  tactical 
activities,  like  contracting,  were  done  in  haste;  Treasury  Board  regulations  and  PWGSC 
contracting  guidelines  were  not  well-understood  by  Provincial  and  Municipal  partners.. 
Originally  local  security  partners  were  negotiating  for  federal  resources,  but  they  were 
unprepared  when  the  contracting  process  took  a  lot  of  time  and  they  received  massive  bills 
for  the  use  of  these  assets.  The  DM  suggested  that  the  process  should  be  changed  so  that 
requests  were  made  from  federal  rather  than  local  security  partners.  When  this  was  done, 
the  cost  of  many  security  aspects  (e.g.,  renting  a  CF  armoury)  went  down  to  nothing  and 
the  timelines  for  approval  decreased; 

•  Over  time  it  became  clear  to  the  DM  that  vital  information  was  not  known  (e.g., 
knowledge  of  the  legal  case  Regina  vs.  Knowlton,  a  case  where  the  finding  resulted  in  the 
authority  to  have  cordons  and  protect  them  when  countries  are  hosting  internationally 
protected  persons).  The  DM  generally  decided  to  inform  people  about  the  information  (DM 
felt  it  would  be  remiss  not  to),  but  then  the  DM  was  marginalized  because  of  this  by  some 
people  because  who  felt  the  DM  was  “sticking  their  nose  in”.  This  caused  concerns  for 
inteipersonal  relationships  and  also  conflicts  between  others  because  some  people  would 
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agree  with  the  DM  and  some  would  disagree,  and  the  implications  of  these  incidents  built 
over  time; 

•  The  individual  agendas  of  security  personnel  involved  played  a  role  in  how  events 
unfolded.  Certain  individuals  were  trying  to  keep  information  to  themselves  (which  led  to 
delays  in  needed  information  being  communicated),  others  appeared  to  have  other 
priorities  than  maintaining  security  for  the  event,  etc.; 

•  Personality  played  a  role  and  the  interactions  between  personality  and  events  changed  over 
time; 

•  There  were  several  C2  cells  actively  coordinating  during  the  event.  .  The  DM  was  present 
in  the  LCC  and  the  “main”  HQ  was  located  elsewhere,  so  this  created  a  high  demand  on 
the  DM  for  information  as  the  DM  was  the  person  “on  the  ground”  with  access  to 
information  about  events  occurring  at  the  international  event.  As  events  unfolded,  the 
pressure  became  more  and  more  intense; 

•  The  demonstrations  and  protests  unfolded  over  time,  as  well  as  their  implications,  such  as 
how  people  were  handling  the  situation,  how  C2  would  operate,  what  extra  security  forces 
were  available,  etc.; 

•  The  breakdown  of  C2  during  the  event  unfolded  over  time  and  had  profound  effects, 
including  far-reaching  consequences  for  personal  relationships  and  the  relationship 
between  security  stakeholders; 

•  Investigations  are  on-going,  and  these  investigations  will  unfold  over  time  and  have  many 
consequences  for  organizational  and  personal  relationships,  organizational  reputation, 
people’s  careers,  recommendations  for  future  events,  etc.  As  the  understanding  that  there 
would  likely  be  investigations  related  to  this  event  grew,  this  influenced  decision-making 
and  some  individual’s  decision  making  became  more  and  more  self-protective. 

2.  Scenario  Description:  NSE  Officer,  Afghanistan 

It  was  the  responsibility  of  the  decision  maker  (DM)  to  manage  logistics  to  support  military 
operations  performed  by  the  CF  in  Afghanistan.  This  involved  managing  a  number  of  physical 
resources  (e.g.,  food,  artillery,  vehicles,  and  other  equipment)  in  terms  of  quantity,  location, 
storage,  maintenance,  etc.  Due  to  a  strategic  emphasis  of  combat  and  de-emphasis  of  logistics,  as 
well  as  the  fact  that  only  a  finite  amount  of  money  was  made  available  to  conduct  the  mission, 
logistics  staff  and  other  resources  were  extremely  limited.  The  Concept  of  Operations  (CONOPS) 
required  these  resources  to  be  dispersed  throughout  the  area  of  interest  (i.e.,  decentralized),  which 
in  turn  required  the  building  and  supply  of  Forward  Operating  Bases  (FOBs).  The  number  and 
locations  of  the  FOBs  were  decided  based  on  anticipated  strategic  and  operational  requirements  - 
the  DM  had  no  input  into  these  decisions.  Due  to  the  fact  that  the  amount  of  resources  was 
extremely  limited,  the  dispersed  physical  location  of  resources  combined  with  the  difficulty  of 
moving  resources  from  one  location  to  another  (locations  were  far  apart  through  rough  terrain 
inhabited  by  the  enemy;  there  were  no  air  assets  available  to  the  CF  other  than  for  emergency 
medical  evacuation)  made  the  logistical  situation  extremely  vulnerable  to  unexpected  events. 

Factors  which  influenced  the  ability  of  the  DM  to  manage  the  resources  included: 

•  Individual  differences  (e.g.,  some  commanders  used  more  artillery  than  others); 

•  Actions  of  the  enemy  (e.g.,  the  number  of  attacks,  the  location  of  attacks); 
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•  The  relative  importance  of  logistics  considerations  in  the  CF  is  quite  variable  and 
frequently  little  effort  is  directed  at  the  sustainment  estimate;  Time  lags  between  requesting 
and  receiving  replenishment  of  resources  (e.g.,  some  critical  operational  stocks  require 
over  1  month  to  order  and  receive  in  a  deployed  setting;  Need  to  work  within  resource 
limitations  (e.g.,  only  a  limited  staff); 

•  Orders  from  higher  command  (e.g.,  orders  to  conduct  operations  in  new  locations); 

•  Resources  of  allies  (e.g.,  the  need  to  share  resources  with  allies)  for  tactical  effect  can 
sometimes  compromise  resupply  of  other  Canadian  units.  Failing  to  assist  allies  with 
material  support  can  mean  denial  of  emergency  access  to  allied  resources;  Strengths  of 
relationships  between  allies  (e.g.,  if  we  help  someone  they  will  be  more  likely  to  help  us  in 
return;  have  we  helped  them  in  the  past;  have  they  helped  us); 

•  Lack  of  information  to  support  planning  as  this  was  a  relatively  different  situation  than 
recent  CF  missions  (e.g.,  this  was  a  hando  ver  of  fighting  units  which  had  not  been  done 
since  the  Korean  war;  there  was  a  lack  of  information  about  how  many  resources  such  as 
ammunition  were  actually  required  as  this  was  a  peace-making  rather  than  peacekeeping 
mission); 

•  Political/strategic  concerns  (e.g.,  it  was  not  permitted  for  the  CF  to  buy  ammunition  from 
some  allies);  and, 

•  Environmental  effects  (e.g.,  maintenance  had  to  be  performed  frequently  because  sand 
interfered  with  vehicle  operation). 

Factors  which  unfolded  over  time: 

o  The  enemy  is  constantly  evolving  their  tactics  (e.g.,  their  ambush  locations  and  methods). 
This  means  that  the  location  of  resources  (e.g.,  FOBs)  became  more  or  less  relevant 
depending  on  enemy  locations  and  activity; 

o  Locations  and  number  of  FOBs  strongly  influenced  logistics  but  they  were  planned  without 
logistics  input  and  had  to  be  built  well  before  they  were  needed;  there  was  no  way  to 
influence  where  they  were  or  know  that  they  would  be  appropriately  situated.  As  events 
unfolded,  the  FOBs  were  not  useful  as  the  CF  were  sent  to  another  part  of  Afghanistan  to 
fight; 

o  The  relative  importance  of  logistics  considerations  in  the  CF  is  quite  variable  and 

frequently  little  effort  is  directed  at  the  sustainment  estimate.  Because  more  firepower  was 
desired  in  the  Afghanistan  mission  but  the  budget  was  limited,  there  was  an  increase  in  the 
proportion  of  combat  to  logistics  personnel.  This  meant  that  there  were  relatively  few 
logistics  personnel  sent  to  support  the  combat  troops,  which  had  both  immediate  and  long¬ 
term  effects  on  the  ability  of  the  logistics  staff  to  keep  up  with  resource  needs  (e.g., 
impacted  psychological  well-being  and  safety  of  logistics  staff  due  to  lack  of  sleep  and 
rest,  impacted  number  of  convoys  which  could  occur,  impacted  the  rate  at  which  items 
could  move,  impacted  the  ability  of  the  staff  to  do  maintenance); 

o  Another  impact  of  directing  little  effort  at  the  sustainment  estimate  was  that  during 

training,  true  sustainment  was  not  practiced  (e.g.,  soldiers  could  go  to  the  base  and  get  food 
easily).  This  led  to  a  lack  of  advanced  information  for  planners  about  true  logistical  needs 
(information  had  to  be  gathered  over  time  during  the  operation)  as  well  as  a  lack  of  training 
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for  soldiers  regarding  true  sustainment  demands  and  readiness  requirements,  which  likely 
impacted  their  morale  and  preparedness; 

o  The  location  of  the  conflict  (Afghanistan)  had  a  profound  impact  on  logistics.  Afghanistan 
is  surrounded  by  other  countries  (i.e.,  no  sea  access);  thus,  ammunition  has  to  be  flown  in. 
Because  of  the  nature  of  artillery,  flying  artillery  over  other  countries  requires  their 
consent,  and  therefore  much  preplanning  is  required  to  move  ammunition  (e.g.,  it  can  take 
over  a  month  to  receive  ammunition  from  Canada).  This  in  turn  results  in  a  lack  of 
flexibility  in  the  ammunition  ordering  process  and  a  resulting  vulnerability  in  the  flow  of 
resources; 

o  There  was  a  radical  change  in  the  CONOPS  for  the  Afghanistan  mission  which  occurred 
immediately  prior  to  the  DM  landing  in  Afghanistan.  This  CONOPS  required  changes  in 
plans  to  how  the  LAV  companies  would  be  used  (i.e.,  LAV  companies  were  always  in  use 
rather  than  having  one  on  base).  This  change  in  resource  allocation  had  long-term 
repercussions  to  how  maintenance  could  be  scheduled  and  performed  (e.g.,  all  LAVs 
constantly  being  out  in  use  means  no  preventative  maintenance  was  possible); 

o  The  use  of  ammunition  varied  greatly  over  time.  Monitoring  ammunition  use  at  the 

beginning  of  the  DM’s  tour  indicated  that  X  amount  was  used  per  unit  time;  however,  this 
increased  greatly  when  the  pace  of  battle  increased  as  well  as  when  other  factors  changed 
(commanders  changed  and  some  commanders  were  more  prone  to  use  a  lot  of 
ammunition).  Factors  which  increased  the  use  of  resources  (especially  ammunition)  also 
had  other  effects  such  as  making  it  more  difficult  for  the  BG  to  report  ammunition  use  (i.e., 
as  the  pace  of  battle  increased,  the  use  of  ammunition  greatly  increased,  but  the 
information  from  the  BG  about  how  much  artillery  was  being  used  was  not  sent;  this  could 
have  been  due  to  the  stress  and  time  pressure  of  battle  or  other  factors); 

o  Resource  availability  was  so  close  to  the  minimum  required  that  it  had  to  constantly  be 
balanced  and  plans  changed;  factors  such  as  vehicles  needing  repair  would  mean  that 
rebalancing  was  required  frequently. 

3.  Scenario  Description:  Liaison  Officer,  Afghanistan 

The  DM  was  assigned  to  act  as  a  liaison  officer  in  the  operations  coordination  centre  in  Kandahar, 
Afghanistan.  It  was  the  responsibility  of  the  DM  to  act  as  a  liaison  between  the  Canadian  Forces 
(CF)  and  the  Afghanistan  National  Army  (ANA),  Afghanistan  National  Police  (ANP),  Kandahar 
Prison,  Border  Security,  and  the  National  Directorate  of  Security  (NDS;  the  Afghanistan  secret 
service).  At  the  beginning  of  the  DM’s  deployment,  there  was  a  massive  escape  from  the  Kandahar 
prison.  This  created  an  atmosphere  of  uncertainty  about  the  level  of  security  present  in  Kandahar, 
and  part  of  the  DM’s  role  was  to  facilitate  an  increase  in  security.  In  particular,  the  DM  was 
required  to  liaise  with  the  relevant  parties  to  increase  security  in  preparation  for  voter  registration 
which  was  to  occur  approximately  8  months  later.  The  DM  was  to  create  a  “Kandahar  city  security 
network”;  had  to  convince  the  relevant  stakeholders  that  it  was  necessary,  persuade  them  to  take 
part,  and  oversee  the  process. 

Factors  which  influenced  the  ability  of  the  DM  to  manage  these  processes  included: 

•  The  prison  break  was  believed  to  be  facilitated  by  assistance  from  inside  the  prison.  Many 
of  the  senior  leadership  at  the  prison  were  either  arrested  or  fired.  This  caused  a  massive 
change  in  personnel  and  a  huge  loss  of  confidence  in  the  prison  system  and  personnel,  as 
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well  as  a  re-evaluation  of  many  assumptions  held  by  CF  personnel  about  security  in 
Kandahar; 

•  Cultural  differences  between  Afghanistan  and  Canada  are  profound;  they  can  create  a  lack 
of  trust  and  difficulty  with  coordination.  These  include: 

o  Extremely  high  emphasis  on  interpersonal  relationships  in  Afghanistan  (e.g.,  the 
importance  of  personal  relationships  to  Afghans  is  paramount).  This  has  profound 
implications;  for  example,  authority  based  on  position  alone  is  not  “real”  authority 
to  the  Afghans  (obedience  is  based  on  personal  relationships); 

o  Differences  in  the  legal  system  and  widespread  corruption  (e.g.,  people  responsible 
for  the  prison  break  probably  were  not  punished  as  they  could  pay  to  be  released); 

o  A  difference  in  the  idea  of  what  it  is  to  have  a  job  and  the  attendant  responsibility 
(e.g.,  a  police  officer  threw  away  his  phone  because  DM  was  getting  too  many  911 
calls,  even  though  that  was  the  only  phone  the  calls  were  coming  to); 

o  The  Afghans  appeared  to  have  the  perspective  that  they  had  little  control  over 
events  (e.g.,  “what  Allah  wills  will  happen”);  whereas  Canadians  typically  feel  we 
do  have  control  over  events.  This  had  implications;  for  example,  the  Afghans  were 
not  used  to  the  concept  of  practicing  for  a  possible  event  and  were  resistant  to  such 
training 

o  Different  social  standards  (e.g.,  the  DM  never  met  the  wife  of  any  Afghan  the  DM 
worked  with).  The  DM  knew  that  social  relationships  were  important,  but  was  not 
able  to  actually  understand  the  relationships  were  as  the  DM  was  not  privy  to  how 
families  were  interrelated  through  marriage; 

o  Literacy  is  so  low  in  Afghanistan  that  written  records  are  rarely  kept,  which  means 
data  is  not  available  that  would  help  assess  resource  and  training  requirements; 

o  Lack  of  appreciation  for  resources  (e.g.,  communications  equipment  was  given  to 
the  relevant  stakeholders;  one  faction  of  the  ANA  changed  locations  but  left  the 
equipment  in  the  old  location  unattended  without  notifying  the  DM); 

o  Age  is  more  of  a  factor  in  creating  a  sense  of  authority  in  Afghanistan.  The  DM 
felt  that  he  would  have  received  more  respect  if  he  had  been  older; 

•  Communication  is  difficult;  communication  systems  are  incomplete  and  there  are  few 
communication  protocols  within  Afghan  organizations; 

•  The  ANA  and  the  ANP  have  an  adversarial  relationship  and  do  not  work  together  well.  It 
might  have  been  an  adequate  solution  for  Kandahar  security  to  just  create  a  system 
between  the  ANA  and  the  ANP;  however,  the  animosity  between  them  was  just  too  great 
to  only  work  with  those  2  organizations; 

•  There  was  a  quick  turnover  in  Afghan  personnel;  for  example,  within  9  months  there  were 
3  governors  of  Kandahar; 

•  The  DM  had  to  ensure  that  he  did  not  insult  the  Afghans  he  was  working  with  (e.g.,  while 
training  them  he  had  to  be  careful  not  to  imply  that  they  didn’t  know  how  to  do  their  jobs); 

•  The  DM  had  to  balance  the  need  to  keep  people  invested  versus  giving  honest  feedback  to 
improve  performance; 
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•  Mentors  had  to  be  persuaded  to  support  the  DM’s  goals.  Mentors  to  the  various  Afghan 
organizations  (e.g.,  RCMP  Personnel  mentored  the  ANP)  provided  guidance  to  the  Afghan 
personnel  and  additional  support  to  the  DM.  The  mentors  often  could  persuade  the 
Afghans  of  the  importance  of  training  etc.; 

•  Sometimes  official  positions  are  not  followed  in  actuality.  For  example,  the  DM  was 
informed  that  he  had  deployment  authority  for  the  ANA  quick  reaction  force,  but  he  tried 
to  deploy  them  three  times  and  all  three  times  he  was  denied; 

•  There  is  a  lack  of  reliable  power  in  Afghanistan;  this  means  that  the  use  of  communications 
equipment  is  limited  by  power  outages  and  often  the  communications  equipment  is  turned 
off  to  conserve  power. 

Factors  which  unfolded  over  time: 

•  There  was  a  9 1 1  system  that  was  implemented;  this  was  supposed  to  be  run  by  the  ANP 
but  the  chief  of  police  discarded  the  phone  that  was  used  for  the  911  calls  as  he  found  it 
inconvenient.  The  DM  volunteered  to  take  over  the  911  function;  this  became  a  tool  to  get 
the  ANP  involved  as  the  operations  room  would  receive  the  calls  but  the  ANP  would  be 
contacted  to  answer  them.  This  created  additional  (and  unpredictable)  windows  of 
opportunity  for  collaboration  with  the  ANP,  for  obtaining  intelligence,  and  for  facilitating 
relationships  with  civilians; 

•  The  prison  break  led  to  a  process  of  information  gathering.  This  process  gradually  revealed 
that  there  was  no  Common  Operating  Picture  (COP)  among  Afghan  security  agencies;  that 
the  ANP  could  not  effectively  get  information  about  threats;  that  the  ANP  had  difficulty 
responding  to  threats;  that  the  ANP  could  not  effectively  ask  for  assistance  from  other 
security  organizations;  and  that  Afghan  security  organizations  typically  work  at  a  tactical 
level  and  are  not  used  to  working  at  an  operational  or  strategic  level.  These  pieces  of 
information  changed  how  the  DM  saw  their  goals  and  how  they  could  be  accomplished; 

•  The  DM  attempted  to  get  the  relevant  stakeholders  to  install,  maintain,  train  on,  and  use 
communication  equipment  provided  by  the  U.S.  Fie  found  this  very  difficult  for  many 
reasons  (e.g.,  the  cultural  differences  listed  above),  and  felt  that  any  intervention  he  tried 
did  not  result  in  change  (i.e.,  there  seemed  to  be  a  “set  point”  in  the  environment  that  was 
highly  resistant  to  change).  This  became  more  and  more  frustrating  as  time  went  on  and  the 
DM  grew  increasingly  hopeless  that  the  C2  infrastructure  would  be  adequate  to  maintain 
security. 

4.  Scenario  Description:  CoE  Training  Developer 

The  DM  was  a  trainer  at  the  Centre  of  Excellence  (CoE),  at  the  Canadian  Land  Force  Command 
and  Staff  College  (CLFCSC).  It  was  the  responsibility  of  the  DM  to  support  the  training  function  of 
the  CoE.  Duties  included  designing  training  packages  and  putting  arrangements  in  place  to  deliver 
them.  The  DM  had  to  incorporate  multiple  components  into  the  training  program  including  cultural 
information,  working  with  translators,  team  training,  interacting  with  the  media  (e.g.,  giving 
interviews),  working  with  multiple  OGDs  (e.g.,  DFAIT,  CIDA),  dealing  with  unexpected  events, 
and  how  to  balance  conflicting  objectives  (e.g.,  when  CEFCOM  and  NATO  have  different  policies 
or  priorities).  The  DM  evaluated  what  worked  and  what  didn’t  work  in  training  and  attempted  to 
improve  the  training  program.  The  DM  also  acted  as  a  liaison  with  non-military  actors  who  would 
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be  taking  part  in  training.  The  DM  had  been  involved  from  the  beginning;  at  the  time  of  the 
interview  they  were  working  on  their  5th  training  cycle. 

Factors  which  influenced  the  ability  of  the  DM  to  manage  these  processes  included: 

•  Flaving  responsibility  but  no  authority;  the  CF  makes  the  decisions  and  the  DM  can  only 
make  requests; 

•  Large  amounts  of  unpredictability  about  how  trainees  will  respond  to  events  in  exercises, 
both  in  terms  of  what  they  will  choose  to  do  and  also  how  well  they  will  handle  events; 

•  Difficulty  evaluating  actions  and  determining  what  likely  results  of  trainee  actions  would 
be.  One  strategy  used  is  to  have  observer/controller  teams  during  exercises  to  observe 
people  and  give  feedback  about  how  people  are  performing.  Usually  these  observers  have 
just  come  back  from  theatre  so  they  have  recent  experience; 

•  Challenges  in  creating  training  exercises  etc.  that  are  both  realistic  enough  and  controlled 
enough  (e.g.,  present  and  understood  2nd  and  3rd  order  effects  for  exercises).  For  example, 
in  Afghanistan,  no  one  in  C1DA  can  tell  you  that  the  Dhala  dam  project  will  make  things 
better  for  the  Afghans;  more  irrigation  could  result  in  people  growing  more  opium  which 
would  be  negative.  Because  it  is  difficult  to  understand  what  consequences  of  events  would 
be  it  is  difficult  to  create  realistic  training  scenarios,  etc.  One  strategy  used  is  to  involve 
SMEs  in  exercise  creation; 

•  Problems  with  creating  believable  training  exercises.  Sometimes  things  that  actually 
happened  in  operations  are  not  seen  as  believable  in  training,  therefore  reality  has  to  be 
balanced  with  the  perception  of  reality  so  that  trainees  will  “buy  in”  to  the  training  scenario 
(the  DM  noted  that  many  of  these  “unbelievable”  events  actually  happen;  when  trainees 
were  questioned  by  the  DM  after  they  come  back  from  their  deployment  they  said  that 
things  happened  that  they  wouldn’t  have  believed  in  training); 

•  Some  aspects  of  training  (e.g.,  training  people  to  give  interviews)  are  difficult  to  mentor; 

•  Resource  requirements  are  often  not  easily  available  (e.g.,  media  representation,  cultural 
component  to  incoiporate  into  scenarios)  and  have  to  be  built  from  scratch; 

•  Changes  in  organization.  As  the  need  for  collaboration  between  different  organizations 
became  understood,  the  Privy  council  office,  Afghanistan  Task  Force  took  over  so  that 
only  one  request  for  participation  had  to  be  made  rather  than  the  DM  having  to  deal  with 
all  organizations  individually; 

•  Trainees  with  limited  time.  There  are  generally  a  great  many  demands  on  trainees’  time, 
and  when  there  are  conflicts  often  training  is  secondary  in  priority  and  therefore  missed. 

For  example,  trainees  have  been  required  to  perform  a  recce  during  the  time  that  a  training 
seminar  had  been  set  up,  and  so  missed  the  training; 

•  Vague  direction.  When  the  training  program  began  initially,  the  DM  was  working  “in  a 
void”,  had  only  limited  staff  and  didn’t  really  know  what  they  needed; 

•  Lack  of  personal  knowledge  about  training  situation.  The  DM  had  never  experienced  the 
kind  of  meeting  that  they  were  trying  to  recreate  in  training,  which  made  it  more  difficult 
to  understand  what  they  were  trying  to  do; 
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•  Feedback  about  training  came  in  several  forms,  including  trainer  intuition  and  feedback 
from  trainees  when  they  came  back  (“we  should  have  known  about  X”).  In  one  case,  the 
DM  saw  an  interview  with  someone  they  had  trained  and  it  was  not  well  done,  which  led  to 
further  training  in  how  to  give  interviews. 

Factors  which  unfolded  over  time: 

•  As  operational  environments  and  priorities  changed,  training  had  to  change  along  with 
them.  This  required  almost  constantly  updating  SA  about  operational  environments  and 
priorities  as  well  as  implementing  changes  to  effectively  deal  with  changes.  Someone  from 
the  CoE  often  went  on  the  recce  with  the  new  HQ  so  they  knew  what  was  needed  in  theatre 
and  what  the  current  situation  was; 

•  The  degree  to  which  the  CF  had  to  work  with  CIDA,  DFAIT,  and  other  organizations  only 
became  clear  over  time  (e.g.,  the  Manley  panel  report  indicated  a  greater  need  for 
cooperation),  therefore  their  involvement  in  providing  and  receiving  training  evolved  over 
time.  There  were  several  components  to  this,  including: 

o  It  took  time  to  build  the  necessary  trust  between  agencies  (e.g.,  CF,  CIDA,  DFAIT) 
to  effectively  train  together.  Networking  and  relationship  building  were  more 
actively  encouraged  over  time  (e.g.,  a  “night  out  dinner”  was  built  into  OGD 
training  to  facilitate  networking  etc.); 

o  Another  aspect  of  OGD  integration  which  took  place  over  time  was  getting  a 
common  understanding  of  basic  Operational  Planning  Process  (OPP).  Initially 
CIDA,  DFAIT  etc.  have  no  training  in  the  OPP  and  this  basic  training  was  begun. 
This  training  helped  improve  common  understanding  and  build  social  networks 
which  were  useful  for  training  and  also  in  theatre.  The  DM  said  that  there  were 
many  2nd  and  3rd  order  effects  from  these  social  relationships  and  other  training 
benefits; 

•  The  money  available  for  training  decreased  over  time,  limiting  opportunities  and  requiring 
adjustments  to  plans.  At  first,  when  the  war  started,  the  budget  was  virtually  unlimited,  but 
funding  became  more  scarce  as  time  when  on  and  the  mission  began  to  wind  up,  the 
recession  started,  etc.; 

•  Generally  training  programs  use  rolling-wave  planning  (i.e.,  planning  in  stages)  to 
accommodate  different  opportunities  as  there  are  many  variables  involved  and  things 
change  over  time  (e.g.,  people’s  availability,  training  needs); 

•  Networks  of  people  were  built  as  the  training  was  developed,  and  this  helped  in  the 
acquisition  of  resources  for  training  as  well  as  getting  other  organizations  involved.  One  or 
two  people  would  get  on  board  (e.g.,  one  member  of  DFAIT)  and  then  their  contacts  were 
useful  for  getting  more  people  involved.  Personal  contacts  often  proved  much  more  useful 
than  “official”  channels; 

•  Unlike  in  earlier  training  cycles,  in  later  cycles  trainers  were  often  trainees  from  an  earlier 
group.  This  meant  that  they  generally  understood  what  they  needed  to  talk  about  and  could 
improve  training,  and  also  decreased  planning  times; 

•  Changes  to  training  programs  had  a  distinct  pattern.  Changes  were  difficult  to  make  within 
a  training  cycle,  but  there  was  generally  a  great  deal  of  change  that  happened  between 
training  cycles. 
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Scenario  Description:  Chief  of  Staff,  Strategic  Advisory  Team, 
Afghanistan 

The  DM  was  the  Chief  of  Staff  for  the  Strategic  Advisory  Team,  Afghanistan,  located  in  Kabul.  It 
was  the  responsibility  of  the  DM  to  bring  generic  planning  skills  to  assist  high  levels  of  the  Afghan 
government  (e.g.,  ministers  and  their  staffs);  he  was  in  charge  of  the  day-to-day  administration 
related  to  this  effort.  Events  which  the  DM  had  to  respond  to  include  the  reaction  to  the  death  of  a 
known  co-worker  in  an  IED  attack,  protests  related  to  the  Danish  cartoon  about  Mohammed,  a 
traffic  accident  which  resulted  in  riots,  a  negligent  discharge  by  one  of  his  personnel,  and  an 
information-gathering  trip  planned  by  some  of  his  staff  which  was  not  well  received  by  Task  Force 
Afghanistan  (TFA)  command. 

Factors  which  influenced  the  ability  of  the  DM  to  manage  these  events  included: 

•  Difficulty  in  getting  accurate  information  in  a  timely  way,  as  there  was  no  access  to  news 
reports  or  “ground  truth”; 

•  The  ability  of  the  Afghan  population  to  communicate  information  quickly  (most  Afghans 
have  cell  phones),  which  made  riots  more  likely; 

•  Communications  back  to  Canada  and  down  to  TFA  were  tenuous.  The  estimate  was  that 
the  DM  was  out  of  touch  with  either  Canada  or  TFA  for  at  least  one  full  day  a  week; 

•  The  location  of  the  DM’s  compound  (close  to  the  US  embassy),  which  meant  that  they  had 
support  close  by  but  also  were  close  to  any  riots  which  were  targeting  the  Americans; 

•  Poor  roadway  infrastructure.  Roads  were  often  poor,  and  information  about  their 
passability  was  not  gained  before  a  major  trip  occurred.  This  created  problems  with 
finishing  the  trip  and  getting  personnel  back  to  the  compound; 

•  Poor  communication  infrastructure.  For  example,  a  cell  network  was  used  to  communicate, 
but  when  voice  traffic  increased  it  was  not  always  possible  to  contact  people  by  voice 
(although  texting  generally  remained  functional); 

•  The  availability  of  only  a  limited  number  of  vehicles  so  travel  required  a  lot  of 
coordination; 

•  Differences  in  culture  that  made  it  difficult  to  coordinate  with  the  Afghans: 

o  “Afghan  time”  meant  that  people  were  often  late; 

o  Many  times  agreements  were  reached  but  then  it  became  clear  that  the  Afghans  did 
not  support  the  agreement,  so  agreements  had  to  be  renegotiated; 

o  Afghans  did  not  do  things  the  way  that  the  DM  and  the  DM’s  team  members 
expected;  for  example,  the  senior  staff  of  governmental  agencies  often  “didn’t 
know  how  to  run  a  meeting”; 

o  Records  were  often  not  kept.  For  example  there  was  no  log  of  visitors  to  the 
president  or  briefings  to  the  president  ahead  of  state  visits; 

o  The  DM  had  to  perform  administrative  functions  such  as  renting  buildings  but 
there  was  basically  no  banking  system  in  Afghanistan; 

•  Ambiguous  direction.  The  goal  of  the  mission  was  to  “do  it  their  way”,  but  it  was  a  process 
to  understand  what  that  meant; 
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•  The  presence  of  a  lot  of  different  organizations  in  Kabul  at  the  time  (including  the 
American  coalition  operation  groups,  ISAF  mission  groups,  UN  mission  and  UN  agencies, 
individual  nations  with  their  embassies  and  ambassadors)  and  their  agendas  were  not 
always  in  line; 

•  The  organization  of  a  daily  meeting  for  the  team  to  coordinate  and  make  sure  that  everyone 
knew  what  was  going  on,  that  people  had  transportation,  etc.; 

•  Building  relationships  with  chaplains  at  a  nearby  US  base.  The  chaplains  acted  as  a 
resource  when  Canadians  needed  counselling  in  difficult  situations  (e.g.,  when  a  known  co¬ 
worker  was  killed); 

•  Afghan  security  was  deployed  outside  the  wall  of  the  DM’s  compound.  This  meant  that  the 
armed  people  outside  the  wall  were  of  the  same  nationality  as  protestors  or  rioters  and  so 
their  presence  was  less  inflammatory  than  armed  Canadian  military  would  likely  have 
been; 

•  Communication  with  team  members.  Members  of  the  team  did  not  always  appropriately 
communicate  information  to  the  DM  in  a  timely  way.  For  example,  when  contacted  by  the 
team  member  who  had  had  the  negligent  discharge,  he  didn’t  inform  the  DM  promptly  that 
he  had  not  informed  his  direct  supervisor  about  the  incident; 

•  The  command  structure.  The  mission  required  a  mission  command  approach  (which  was 
used)  rather  than  an  overly  hierarchical  C2  approach  because  flexibility  was  required  and 
individuals  had  to  exercise  initiative  to  effectively  conduct  the  mission. 

Factors  which  unfolded  over  time: 

•  One  goal  of  the  DM  was  to  build  relationships;  relationship  networks  were  built  over  time 
and  had  many,  sometimes  unexpected  benefits.  For  example,  the  DM  made  contact  with 
the  chaplains  at  a  nearby  US  base,  and  these  chaplains  acted  as  a  resource  when  Canadians 
needed  counselling  (e.g.,  when  their  co-worker  was  killed).  To  facilitate  relationship 
building,  the  DM’s  group  hosted  an  open  house  BBQ  every  Friday  night,  and  as 
relationships  built  more  and  more  people  would  attend.  This  created  a  growing  network  of 
connections  between  people  and  facilitated  many  interactions; 

•  There  was  an  on-going  balancing  act  between  being  openly  military  (e.g.,  wearing 
uniforms,  driving  military  vehicles)  and  trying  to  pass  as  civilian,  which  were  influenced 
by  factors  such  as  the  current  state  of  tension  in  the  area.  The  costs  (e.g.,  being  targeted, 
legal  trouble  and  safety  issues  if  caught  with  weapons  out  of  uniform)  and  benefits  (e.g., 
being  protected  in  more  secure  vehicles,  being  protected  by  military  law  as  on-duty 
military  officers)  have  to  be  weighed.  This  had  far-reaching  consequences,  including 
strategic  implications  if  other  country  members  saw  Canadian  military  entering  Afghan 
military  buildings; 

•  The  chain  of  command  was  fuzzy.  Task  Force  Afghanistan  (TFA)  in  Kandahar  was  in 
charge  of  the  DM’s  group  for  administrative  purposes,  but  the  team  actually  worked 
directly  through  CEFCOM.  As  time  went  on  it  became  more  and  more  clear  that  TFA  felt 
they  had  more  authority  over  the  DM’s  team  than  the  DM  thought,  and  this  made  decision 
making  more  difficult.  For  example,  TFA  kept  pressuring  the  DM  to  get  electronic 
counter-measures  put  on  their  vehicles,  but  the  DM  kept  putting  this  off  as  that  would 
make  them  targets  (because  it  would  be  obvious  they  were  military  vehicles); 
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•  Team  cohesion  issues  waxed  and  waned  during  this  time  period.  For  example,  the  team 
member  who  experienced  the  negligent  discharge  was  teased  about  it  by  the  group  for 
some  time  afterward.  The  group  lived  together  and  had  to  be  a  tight-knit  group,  so  any 
tensions  could  profoundly  impact  the  mood  of  the  entire  compound  and  the  ability  of  team 
members  to  work  effectively  together.  The  DM  experienced  2  different  team 
configurations  and  the  second  configuration  included  more  team  players  than  the  first, 
which  brought  home  the  importance  of  this  factor; 

•  Events  which  threatened  the  security  of  the  compound  led  to  reassessment  of  the  security 
plan  for  the  compound  as  well  as  an  on-going  effort  to  work  with  the  British  and  other 
embassies  to  arrange  for  a  more  secure  location  in  case  an  evacuation  was  required,  and 
also  an  attempt  to  improve  communication  between  these  stakeholders. 

6.  Scenario  Description:  PME  Revitalization  Supervisor 

It  was  the  responsibility  of  the  decision  maker  (DM)  to  oversee  and  evolve  the  Professional 
Development  (PD)  system  and  to  ensure  that  CF  schools  were  implementing,  conducting,  and 
delivering  education  and  training  as  laid  out  according  to  the  PD  system.  When  entering  the 
position,  the  DM  became  aware  that  there  was  an  on-going  issue  with  Professional  Military 
Education  (PME)  revitalization.  This  file  had  been  sitting,  not  moving  forward,  for  18  months.  The 
main  issue  appeared  to  be  getting  funding  approval  for  the  program.  There  appeared  to  be  a 
fundamental  disagreement  between  one  CF  educational  institution  (referred  to  as  “School”)  and  the 
staff  from  another  organization  who  were  supposed  to  assist  in  creating  a  funding  proposal 
(referred  to  as  “Staff’).  Thus,  the  responsibilities  of  the  DM  included  coordinating  several 
organizations  to  exchange  needed  information  and  managing  disagreements  that  arose  between  the 
different  stakeholders. 

Factors  which  influenced  the  ability  of  the  DM  to  manage  this  process  included: 

•  Disagreements  between  stakeholders  about  the  scope  of  project  financing  (e.g.,  were 
overhead  and  general  infrastructure  upgrades  to  the  School  [e.g.  modernize  classrooms, 
expand  quarters,  etc.]  meant  to  be  included?)  which  led  to  different  estimates  of  the 
funding  requirements; 

•  Stakeholders  not  acting  within  their  range  of  authority  and  mandate  (e.g.,  Staff  were 
questioning  the  value  of  the  PME  Revitalization  [asking  if  it  was  money  well  spent,  etc.] 
when  this  was  not  supposed  to  be  part  of  their  role); 

•  There  were  continual  delays  brought  upon  by  Staff  requiring  clarification  on  small  details; 

•  Difficulty  in  understanding  what  other  people’s  motivations  are  (i.e.  are  there  hidden 
agendas); 

•  Periods  of  silence  of  significant  duration  (e.g.,  several  weeks  or  more)  often  occurred  after 
submission  of  requested  information  to  Staff; 

•  Requirements  for  new  or  expanded  educational  programs  often  do  not  come  with 
additional  funds,  requiring  a  shifting  of  resources; 

•  Requests  for  clarification  from  superiors  about  what  programs  are  a  priority  (and  therefore 
which  can  be  reduced  or  eliminated  to  fit  budget  constraints)  do  not  result  in  clear 
direction; 
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•  A  major  Strategic  Review  being  undertaken  that  affected  the  whole  CF  during  this  time 
period.  It  was  anticipated  that  the  results  of  the  review  will  profoundly  affect  funding, 
including  funding  related  to  PME  revitalization.  It  was  believed  that  this  led  some  people 
to  be  reluctant  to  make  decision; 

•  The  requirement  (on  several  occasions)  to  push  something  to  a  higher  level  of  authority  for 
a  decision  or  try  to  get  resolution  (i.e.,  ask  for  clarification  of  Staff  commander  objectives) 
due  to  conflicting  outputs  from  Staff  (i.e.  told  that  this  is  a  priority  for  them,  but  they  do 
not  see  a  need  to  put  members  forward  to  attend  a  2  day  workshop).  The  decision  to  push 
an  issue  up  the  chain  of  command  involves  evaluating  the  risks  involved  (as  it  could  mean 
unnecessary  delays,  etc.); 

•  A  need  to  balance  short-term  and  long-term  gains  in  terms  of  management  style  (e.g., 
enforcing  authority  can  help  to  solve  some  problems  in  the  short  term  but  relying  on 
authority  can  reduce  buy-in  longer  term); 

•  Communication  clarity  (e.g.,  terminology  meant  different  things  to  different  people); 

•  Difficulties  in  organizing  people  to  be  in  the  same  place  at  the  same  time  for  meetings; 

•  The  School  was  told  that  their  funding  would  no  longer  have  the  flexibility  of  receiving  in¬ 
year  funding,  so  resources  would  be  further  limited; 

•  If  an  educational  program  is  being  changed,  the  changes  have  to  be  made  with 
consideration  of  all  environments  (army,  navy,  and  air  force)  to  ensure  that  the  changes  to 
the  system  or  course  are  consistent  with  needs  of  all  elements  and  their  philosophies. 
Because  of  significant  differences  it  is  difficult  to  determine  what  education  is  required, 
how  it  should  be  done,  how  they  are  going  to  get  buy-in  from  different  stakeholders,  etc.; 

•  Changes  in  one  aspect  of  an  education  program  have  to  take  into  account  other  phases  of 
the  program.  For  example,  each  Developmental  Period  (DP)  requires  that  you  meet  certain 
necessities  so  that  the  person  can  move  on  to  the  next  DP  level; 

•  If  something  is  added  to  a  DP,  then  something  else  may  have  to  be  removed  and  this  could 
cause  knock-on  effects  through  the  other  DP  levels  as  other  things  have  to  be  shuffled  out; 

•  The  time  of  those  being  educated  is  limited. 

Factors  which  unfolded  over  time: 

•  Command  direction  slowed  the  achievement  of  goals  and  more  scrutiny  was  paid  as  the 
number  of  small  information  requests  increased.  Whether  there  was  general  risk  aversion 
or  other  factors  were  at  play  is  unknown;  Meeting  requests  were  responded  to  less 
favourably  over  time,  which  delayed  the  process; 

•  The  DM’s  own  staff  had  difficulty  staying  motivated  and  stress  increased  as  the  process 
carried  on  over  time  with  little  apparent  progress; 

•  Lack  of  continuity;  as  old  people  left  and  new  people  entered  (including  the  DM)  the 
process  it  meant  that  tasks  were  performed  multiple  times,  people  had  to  be  gotten  up  to 
speed,  etc.; 

•  The  complexity  of  the  operating  environment  changed  over  time.  The  Training  and 
Education  (T&E)  system  is  meant  to  keep  up  with  changes;  However,  the  T&E  system  was 
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typically  fairly  slow  to  react  to  new  things,  and  so  important  windows  of  opportunity  might 
have  been  missed; 

•  The  rate  at  which  things  changed  was  variable;  for  example,  sometimes  submitting 
information  to  the  Staff  meant  that  the  DM  and  the  DM’s  staff  were  bombarded  with 
requests  for  more  information,  and  sometimes  weeks  would  go  by  with  no  feedback  from 
the  Staff  at  all; 

•  Instructors  had  to  be  hired  to  run  programs;  this  had  to  be  done  far  in  advance  (e.g.,  to  go 
through  PWGSC  process).  Due  to  delays  in  the  PME  process,  opportunities  to  have 
particular  instructors  at  particular  times  were  likely  missed,  and  decisions  had  to  be  made 
without  proper  support  from  a  finalized  PME  process. 

7.  Scenario  Description:  CoE  Training  Development  Supervisor 

The  DM  was  the  contract  site  leader  at  the  Centre  of  Excellence  (CoE),  at  the  Canadian  Land  Force 
Command  and  Staff  College  (CLFCSC).  It  was  the  responsibility  of  the  DM  to  support  the  training 
function  of  the  CoE.  One  of  the  groups  requiring  training  are  the  members  of  HQs  to  be  deployed 
to  Afghanistan  involving  a  4-week  package  of  training  they  deliver  to  the  HQ  in  their  base  location. 
This  training  development  is  an  on-going,  iterative  process.  There  is  a  need  to  look  forward  by  at 
least  a  year,  both  to  arrange  the  logistics  of  training  (e.g.,  preparations  for  an  exercise  in  February 
have  to  start  by  September  for  an  HQ  deploying  in  July)  and  incoiporate  upcoming  changes  into 
training  programs.  The  current  change  being  considered  is  a  focus  change  from  Counterinsurgency 
(COIN)  operations  to  mission  termination. 

Factors  which  influenced  the  ability  of  the  DM  to  manage  these  processes  included: 

•  There  needs  to  be  overlap  and  continuity  in  training  (e.g.,  one  HQ  has  to  both  take  over 
from  another  HQ  and  set  the  conditions  for  the  next  HQ  to  come  in); 

•  Government  announcements  (e.g.,  moving  main  location  from  Kandahar  to  Kabul)  can 
force  unexpected  readjustments  in  training; 

•  Changes  may  be  announced  but  details  may  not  be  known,  requiring  assumptions  to  be 
made  so  that  enough  lead  time  is  available  to  plan  training; 

•  Training  cannot  be  largely  template  because  the  rate  of  change  is  too  high  (e.g., 
commanders  are  given  different  primary  tasks); 

•  Outside  events  force  a  lot  of  change,  e.g.,  changes  in  policy,  different  events  happening  in 
Afghanistan; 

•  Writing  teams  who  design  the  training  are  experienced  and  generally  understand  what  is 
required; 

•  There  are  often  overlapping  requests  for  training  time  and  resources  that  require  on-line 
resource  shifting; 

•  Long  lead  times  are  often  required  for  training  objectives  to  be  met  (e.g.,  the  new  HQ  has 
new  objectives  to  move  equipment  back  to  Canada,  a  lot  of  equipment  and  containers  have 
to  be  inspected,  inspectors  are  few  and  require  time  to  inspect,  etc.); 

•  Training  of  CF  personnel  also  involve  training  them  to  deal  with  other  actors  including 
members  of  DND,  DFAIT,  CIDA,  Correctional  Services,  Policing  Associations,  NGOs  - 
the  DM  and  his  staff  have  to  put  in  a  lot  of  work  to  get  these  groups  adequately  involved; 
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•  The  different  stakeholders  involved  often  have  different  jargon  and  ways  of  doing  things 
which  make  communication  and  collaboration  difficult; 

•  The  timelines  of  the  stakeholders  are  different.  For  example,  the  CF  focuses  on  short-term 
(e.g.,  6  month  long)  projects  with  high  impact,  whereas  CIDA  focuses  on  projects  with 
much  longer  timelines  (e.g.,  15  years); 

•  Money  has  become  much  more  of  a  constraint  than  previously,  which  affects  the  size  and 
duration  of  the  writing  board  (who  create  scenarios  and  other  training  materials)  and  other 
resources  available; 

•  Often  FlQs  standing  up  have  basic  teamwork  etc.  issues  to  sort  out  (e.g.,  sort  out  SOPs, 
learn  about  each  other’s  working  styles  and  create  a  team)  rather  than  just  training  for  a 
specific  mission; 

•  The  DM  had  a  long-term  relationship  with  the  project  lead  and  therefore  had  a  foundation 
of  trust  that  he  could  use  to  convince  the  lead  of  needed  changes; 

•  Training  scenarios  have  to  be  firmly  grounded  in  the  current  state  in  Afghanistan  (or  other 
area  of  interest  related  to  the  training),  because  often  training  is  being  done  after  the  HQ 
does  an  initial  recce,  so  the  trainees  are  usually  well  aware  of  the  current  situation  and  what 
is  relevant.  Trainers  have  to  have  current  knowledge; 

•  The  DM  had  relationships  with  members  of  some  groups  which  helped  the  DM  get  needed 
personnel  involved  in  writing  boards  and  other  aspects  of  training  (i.e.,  he  used  his  social 
networks); 

•  The  DM  had  been  involved  in  this  training  program  for  some  time  (over  3  years)  and  had 
an  understanding  of  many  aspects  of  what  will  work  and  what  will  not.  For  example,  it  is 
important  to  have  2  exercises  separated  in  time  rather  than  1  exercise,  as  one  needs  to  be  a 
“walk”  and  the  next  needs  to  be  a  “run”.  If  these  are  not  separated  somewhat  in  time  (i.e., 
more  than  a  day  or  two)  there  is  not  enough  time  for  the  trainees  to  absorb  what  they 
learned  from  the  “walk”  session  before  the  next  exercise; 

•  The  request  to  include  logistics  personnel  in  the  training  made  CANOSCOM  aware  that 
more  logistics  preparation  for  the  mission  was  required  (the  DM  was  unaware  that  this  had 
not  been  a  part  of  plans  -  it  was  surprising  and  seemed  to  show  a  lack  of  foresight); 

•  The  DM  and  his  civilian  staff  have  to  have  personalities  which  can  accept  not  having  real 
authority  (i.e.,  the  CF  has  the  final  say). 

Factors  which  unfolded  over  time: 

•  Training  program  changed  over  time  as  different  strategies  were  tried  as  the  opportunity 
arose,  and  what  worked  was  kept  or  improved  and  what  didn’t  work  was  removed  or 
changed; 

•  Early  attempts  were  made  to  integrate  organizations  such  as  DFAIT,  CIDA,  and 
CANOSCOM  into  the  training  process  to  ensure  mutual  awareness  of  methods  and  build 
networks.  Over  time  it  became  apparent  that  these  training  sessions  could  prove  very 
beneficial  so  some  organizations  became  more  involved  relatively  early  (e.g.,  DFAIT). 
When  certain  training  goals  changed  to  become  very  relevant  to  other  organizations  (e.g., 
CANOSCOM),  then  they  became  involved  at  that  point; 
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•  The  personnel  recruited  to  run  the  training  changed  frequently  across  time.  A  new  training 
focus  may  require  a  change  in  SME  personnel.  This  requires  active  recruitment  of 
personnel  with  the  relevant  experience  to  create  a  pool  of  people  who  can  be  drawn  upon 
for  training  needs.  There  is  always  a  need  to  find  recently  retired  people  with  more  current 
experience,  as  people  who  have  been  retired  for  longer  may  have  lost  touch  with  current 
issues,  processes,  etc.  As  well,  for  each  individual  occasion  people  have  to  be  scheduled  in 
and  often  there  are  fluid  schedule  constraints  based  on  other  people’s  priorities  (holidays 
etc.)  and  the  needs  of  other  projects  (e.g.,  preparations  for  the  Olympics  required  a  large 
staff); 

•  The  CF  staff  in  the  training  programs  change  cyclically  and  frequently  over  time  (e.g.,  in 
the  3.5  years  the  DM  had  been  doing  this,  he  had  4  bosses).  So,  certain  things  need  to  be 
done  over  and  over  again  (building  relationships,  making  people  aware  of  how  things  have 
been  done,  what  worked,  what  didn’t,  etc.); 

•  There  was  a  request  to  inject  another  group  (high-readiness  HQ)  into  the  training  schedule. 
This  had  follow-on  effects  such  as  having  to  plan  for  concurrent  training,  having  to  re¬ 
distribute  the  workload,  and  new  people  had  to  be  hired; 

•  The  training  programs  and  operational  plans  inform  one  another  and  both  change  over 
time.  For  example,  the  writing  board  looks  for  weak  or  grey  areas  in  the  plan  to  use  as 
injects  for  training.  This  information  is  then  used  by  the  planners  to  tighten  up  weak  spots 
and  hopefully  avoid  problems. 

8.  Scenario  Description:  HF  Engineer,  Capital  Acquisition 

The  DM  was  a  Human  Factors  (HF)  Engineer  on  a  capital  acquisition  project  for  the  CF.  It  was  the 
responsibility  of  the  DM  to  validate  user  requirements  and  write  HF-based  specifications  for  the 
project.  Goals  of  the  project  included  creating  a  specification  that  was:  easy  to  meet,  met  empirical 
standards,  and  met  user  requirements  including  usability,  as  well  as  meeting  time  and  financial 
resource  requirements.  The  DM  had  to  develop  measures  for  user  requirements  and  determine 
which  form  of  evaluation  to  use  (e.g.,  objective  tests,  a  controlled  usability  trial,  etc.) 

Factors  which  influenced  the  ability  of  the  DM  to  manage  these  processes  included: 

•  Different  team  members  and  the  DM  had  consistently  different  perspectives.  While  the 
DM  thought  getting  user  involvement  was  important,  the  other  team  members  didn’t  want 
user  involvement  in  the  requirements  specification  process; 

•  The  DM  tried  many  different  methods  and  explained  the  importance  of  HF  to  other  team 
members  many  times  but  was  largely  ignored.  This  interpersonal  conflict  created  a  feeling 
that  the  DM  was  doing  everything  they  could  but  nothing  changed; 

•  Insufficient  data  was  available  regarding  certain  aspects  of  the  equipment  needed  to  specify 
HF  requirements 

•  Feedback  from  industry  about  initial  requirements  caused  a  lot  of  assumptions  by  the  DM’s 
team  to  be  questioned; 

•  The  DM  was  involved  in  other  similar  projects  and  attempted  to  do  related  R&D  by 
leveraging  those  projects  to  get  needed  information  as  opportunities  arose; 
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Because  of  previous  work  the  DM  had  a  network  of  interpersonal  relationships  that  could 
be  leveraged  to  get  additional  data; 

The  DM  used  their  own  network  of  interpersonal  relationships  to  push  the  HF  agenda  (e.g., 
got  the  Deputy  Project  Director  who  did  work  on  other  projects  to  talk  to  other  team 
members  about  F1F  issues); 

The  DM  was  not  co-located  with  other  decision  makers.  One  impact  of  this  was  that  the 
DM  was  not  always  consulted  before  major  scoping  or  other  project  changes  were  made; 

It  was  difficult  to  convince  the  project  team  to  use  a  testing  baseline  for  bid  evaluation  (to 
ensure  systems  being  acquired  would  offer  improved  capability  over  in-service  systems,  an 
approach  necessary  in  the  absence  of  performance  data  for  in-service  systems),  because  the 
baseline  equipment  was  in  short  supply,  it  would  impose  a  lot  of  logistic  challenges  to 
make  this  equipment  available  to  bidders,  and  much  of  the  baseline  equipment  was 
composed  of  controlled  goods; 

Users  have  multiple  conflicting  needs.  Changing  equipment  to  address  one  of  these  needs 
will  generally  affect  all  of  the  other  needs  as  well  (either  positively  or  negatively); 

Other  teams  on  the  project  had  hired  staff  too  early  (i.e.,  hired  a  large  engineering  team 
before  the  Statement  of  Requirements  (SOR)  was  finalized)  and  therefore  the  project  was 
going  through  budget  resources  quicker  than  anticipated.  This  resulted  in  a  significant 
spend  early  in  the  project  on  engineering,  with  little  actual  project  advancement,  because 
engineering  specifications  kept  having  to  change  to  reflect  SOR  changes.  It  also  meant 
that  less  money  was  available  for  HF  SOR  and  test  specification  validation  work. 

The  credibility  of  the  HF  engineers  and  their  approach  was  continually  questioned  -  as 
evidenced  by  the  requirement  to  continually  present  arguments  with  supporting  data  to  the 
larger  project  management  team  for  approval  (whereas  the  other  specialty  engineering 
teams  were  not  questioned  by  the  project  management  team) ,  the  requirement  to  provide 
way  too  much  detail  in  bid  evaluation  documentation  (which  later  needed  to  be 
dramatically  revised  and  abbreviated,  at  additional  cost  to  the  project,  to  reflect  the  changes 
in  project  scope),  as  well  as  evidenced  in  changes  made  by  other  team  members  that  were 
readily  accepted  by  project  management  with  little  questioning  or  demands  for  evidence; 

There  was  resistance  to  including  testing  for  all  important  interacting  human  factors;  while 
there  were  thousands  of  engineering  specifications,  HF  criteria  were  continually  being 
eliminated  until  they  numbered  fewer  than  50; 

There  were  concerns  that  there  would  not  be  enough  bidders  because  the  bidders  would  not 
be  willing  to  have  their  systems  subjected  to  usability  testing  comprising  scientifically 
administered  user  ratings  of  acceptability.  The  Engineering  Section  wanted  to  water  down 
the  technical  requirements  that  had  been  empirically  validated  by  the  DM  to  ensure  as 
many  bidders  as  possible,  but  at  the  risk  of  buying  a  system  that  would  not  meet  the 
demonstrated  user  needs.  There  were  no  strategies  put  in  place  to  control  the  number  of 
bidders  and  there  turned  out  to  be  more  bidders  than  could  easily  be  handled  in  bid 
evaluation; 

The  scope  of  the  project  changed  from  a  situation  in  which  the  DM’s  team  would  work 
with  the  winning  bidder  to  develop  and  customize  a  system,  to  a  situation  where  systems 
the  bidders  presented  to  be  evaluated  were  what  the  project  would  buy  for  the  CF  (i.e.,  no 
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CF  customization).  This  shortened  the  project  schedule,  but  created  a  huge  increase  in  risk, 
greatly  increased  the  required  specificity  for  the  statement  of  requirements,  and  made  the 
bid  evaluation  process  much  more  important. 

Factors  which  unfolded  over  time: 

•  The  awareness  of  technology  that  was  available  changed  over  the  course  of  the  project, 
which  influenced  ideas  about  what  was  possible  and  created  different  opportunities; 

•  The  specification  of  requirements  was  an  iterative  process  and  feedback  from  industry 
caused  assumptions  about  available  technology  to  be  re-evaluated  and  changed; 

•  Financial  resource  availability  for  FIF  changed  over  time;  initially  they  were  supposed  to 
have  resources  for  Research  and  Development  (R&D)  but  this  allocation  of  resources 
dropped  over  time  without  needed  R&D  being  accomplished  (i.e.,  it  was  reallocated  for 
other  non-FlF  purposes); 

•  The  project  proved  to  be  quick  to  spend  and  slow  to  deliver;  the  strategic  review  and  other 
forces  caused  scope  changes,  resource  restrictions,  and  pressure  to  deliver; 

•  The  scope  of  the  work  changed  repeatedly  over  time  which  had  impacts  on  the  other 
aspects  of  the  project. 

9.  Scenario  Description:  PME  Revitalization  Staff  Member 

The  DM  was  a  senior  staff  officer  who  supported  Professional  Development  (PD)  for  the  CF.  Part 
of  the  DM’s  responsibilities  was  to  oversee  the  creation  of  policies  and  processes  related  to  pan-CF 
training  programs.  At  the  beginning  of  this  scenario,  there  was  a  demand  from  higher  command  to 
create  a  Distance  Learning  (DL)  option  for  education  and  training,  which  was  associated  with  a 
revamping  of  the  Professional  Military  Education  (PME)  system  (called  PME  revitalization).  The 
DM  had  to  perform  tasks  such  as  organizing  meetings,  putting  together  briefing  packages  for 
meetings,  and  maintaining  communications  between  the  various  stakeholders  to  facilitate  the  PME 
revitalization  process. 

Factors  which  influenced  the  ability  of  the  DM  to  manage  this  process  included: 

•  The  need  for  consultation.  If  a  training  program  is  being  changed,  the  DM  shouldn’t  make 
any  recommendations  that  affect  policies  for  CF  programs  without  consulting  all  elements 
(army,  navy,  air  force); 

•  Changes  to  the  proposal  submission  process.  The  process  required  to  submit  proposals 
related  to  PME  Revitalization  changed  during  this  time  period.  As  the  procedure  was  new, 
there  was  a  general  lack  of  clarity  about  expectations  and  difficulty  in  following  this 
process; 

•  The  Strategic  Review.  A  major  Strategic  Review  was  undertaken  that  affected  the  whole 
CF  during  this  time  period.  It  was  anticipated  that  the  results  of  the  review  will  profoundly 
affect  funding,  including  funding  related  to  PME  revitalization; 

•  Differing  interpretations  between  stakeholders  about  the  scope  of  project  financing  (e.g., 
was  overhead  and  general  infrastructure  upgrades  meant  to  be  included?); 

•  Inaccuracies  in  databases  of  funding  requirements  (there  appeared  to  be  accounting  errors); 
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•  Difficulty  in  isolating  components  of  programs  to  get  accurate  costing  information. 
Proposals  for  funding  generally  involve  many  interrelated  factors  such  as  actual  cost  to 
deliver  a  program,  infrastructure  and  overhead  issues  (e.g.,  creation  of  a  virtual  library  is 
required  for  a  program  but  it  is  also  used  for  other  things  so  how  can  you  determine  what 
program  should  pay  what  proportion  of  the  cost?); 

•  Periods  of  silence  of  significant  duration  (e.g.,  several  months  or  more)  after  submission  of 
requested  information  to  higher  headquarters; 

•  The  need  for  briefings  to  be  thorough  yet  concise  because  those  being  briefed  don’t  have  a 
lot  of  time  to  read  the  documents; 

•  Requirements  for  new  training  programs  which  often  do  not  come  with  additional  funds, 
requiring  a  shifting  of  resources. 

Factors  which  unfolded  over  time: 

•  Improvements  and  changes  in  technology  (e.g.,  the  possibility  of  creating  a  virtual  library) 
put  different  demands  on  and  created  different  opportunities  for  T&E; 

•  As  the  recession  occurred  and  got  more  serious  it  caused  increasing  funding  concerns; 

•  As  the  impact  of  the  Canadian  operation  in  Afghanistan  changed,  other  priorities  increased 
(e.g.,  setting  up  resource  centres  for  returning  soldiers,  supporting  military  families, 
supporting  the  health  and  welfare  of  returning  soldiers)  so  projects  like  the  PME 
revitalization  got  pushed  to  a  lower  priority  and  delayed; 

•  Training  priorities  changed  over  time  as  senior  commanders,  governments,  and 
governmental  priorities  changed  (e.g.,  more  or  less  demand  for  aboriginal  programs); 

•  Things  would  go  quiet  for  a  few  months  and  then  there  would  be  a  flurry  of  activity  (e.g., 
requests  for  more  information)  which  would  require  a  response  and  then  it  would  go  quiet 
again;  this  appeared  to  the  DM  to  be  a  cyclical  process; 

•  Meeting  requests  were  responded  to  less  favourably  over  time,  which  created  delays  in 
scheduling; 

•  Lack  of  continuity,  with  new  people  entering  the  process,  due  to  annual  postings,  meant 
that  tasks  were  performed  multiple  times  as  people  had  to  be  gotten  up  to  speed,  etc. 

10.  Scenario  Description:  PSYOPS  Training  Program  Developer 

It  was  the  responsibility  of  the  DM  to  develop  a  training  program  for  a  PSYOPS  component  of  an 
Afghanistan  deployment.  The  goal  was  to  create  the  best  PSYOPS  platoon  possible,  able  to 
influence  and  disseminate  information  in  a  productive  manner. 

Components  of  training  that  the  DM  was  responsible  for  included  arranging  quarters  and  rations, 
planning  basic  courses  and  the  schedule,  arranging  course  locations,  creating  an  internal 
confirmation  exercise,  arranging  a  2-week  long  final  exercise  in  Alberta,  and  coordinating 
PSYOPS  training  with  the  general  training  requirements  that  also  had  to  be  provided.  Instructors 
had  to  be  identified  and  recruited.  There  was  a  general  concept  plan  “in  pencil”  before  the  DM  took 
over,  but  final  arrangements  were  generally  up  to  the  DM. 

Factors  which  influenced  the  ability  of  the  DM  to  develop  the  training  program  included: 
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•  This  was  the  first  time  such  a  thorough  training  program  was  being  developed  for  PSYOPS 
training  in  Afghanistan  so  previous  courses  were  not  available  to  use  to  guide  planning; 

•  There  was  a  very  short  timeline  as  budget  approval  was  obtained  6  weeks  before  the  4- 
month  training  program  was  to  begin.  Because  of  the  short  timeline,  changes  in  one 
resource  (e.g.,  instructor  availability)  meant  that  things  had  to  be  shifted  (e.g.,  the  order  of 
instruction  changed); 

•  Some  elements  of  planning  were  done  before  the  DM  took  over,  including  preliminary 
arrangements  for  quarters  and  rations.  This  limited  the  DM’s  options  (e.g.,  there  was 
pressure  to  have  the  training  at  the  PSTC  in  Kingston  but  financial  approval  for  rations 
were  given  for  Toronto); 

•  There  was  pressure  to  have  the  trainees  be  able  to  prove  their  worth  right  away  -  they  were 
to  be  handed  off  to  the  BG  very  quickly  after  their  training  ended; 

•  The  trainees  were  a  diverse  group  of  people  with  very  different  backgrounds  and  levels  of 
military  experience  (although  none  were  new  soldiers); 

•  Two  group  members  had  significant  experience  in  close  quarter  combat  training;  this  was 
used  by  the  DM  and  two  weeks  of  close  quarter  combat  training  was  offered  internally. 
This  offered  a  basic  review,  built  trust  within  the  group,  increased  the  confidence  of  the 
group,  and  gave  the  trainees  an  additional  skill  set  to  increase  their  value  to  the  other 
personnel  they  were  stationed  with; 

•  The  DM  was  requested  by  the  training  location  to  not  have  the  trainees  in  uniform; 

•  The  DM  had  a  network  of  resources  that  proved  very  useful  for  providing  training 
opportunities  (e.g.,  he  knew  someone  at  TV  Ontario;  this  created  an  opportunity  for  5 
trainees  to  observe  their  communications  methods); 

•  Locating  the  training  in  Toronto  allowed  access  to  a  population  of  Afghan  immigrants  that 
were  used  to  assist  training  (e.g.,  to  provide  experience  using  translators  and  to  provide 
cultural  information); 

•  Instructor  schedules  produced  constraints  on  when  information  could  be  delivered  (there 
were  a  lot  of  availability  challenges,  as  instructors  were  generally  booked  farther  in 
advance  then  the  lead  time  given  for  this  course).  This  meant  that  sometimes  information 
could  not  be  delivered  in  the  optimal  order. 

Factors  which  unfolded  over  time: 

•  Planning  for  the  training  programming  was  still  underway  during  the  training  itself  (i.e., 
rolling-wave  planning).  For  example  the  close-combat  training  was  arranged  after  the 
group  had  already  started  training  and  the  skills  of  the  trainees  became  known.  This 
allowed  the  DM  to  take  advantage  of  opportunities  that  arose  but  was  not  optimal  for  long¬ 
term  planning; 

•  Group  cohesion  was  an  important  process  for  facilitating  training;  this  process  worked  well 
in  this  situation  and  was  impacted  by  factors  such  as  the  training  centre  requiring  use  of 
civilian  clothes,  trainees  boarding  together,  and  team  members  conducting  close  quarter 
combat  training.  In  this  case,  the  group  gelled  together  well  (teambuilding  was  considered 
to  be  “outstanding”); 
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•  Some  of  the  benefits  of  training  unfolded  over  time  rather  than  being  immediately 

apparent.  For  example,  some  adversarial  intent  theory  was  provided  by  the  DRDC  group. 
The  usefulness  of  this  was  not  immediately  apparent  to  the  trainees  but  the  relevance  and 
importance  was  understood  later  (e.g.,  when  they  had  to  defend  their  ideas  in  the  field  they 
could  call  upon  related  academic  literature  for  support). 
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Annex  C:  Operational  Experience  Mapped  to 
Five  Key  Complexity  Factors 


This  annex  contains  the  operational  experience  mappings  to  the  five  key  complexity  factors  for 
each  scenario.  The  mappings  are  presented  in  order  of  overall  complexity  ranking  (see  Table  10), 
with  the  most  complex  scenario  first.  Note  that  some  specifics  have  had  to  be  removed  due  to 
concerns  about  privacy. 

1.  Experience  to  complexity  factor  mapping:  Military  Liaison  and 
Advisor,  International  Event 


Concept  and  Definition 

Concept  Examples 

Concept  Importance  for 
Scenario  (High,  Medium, 
Low)  and  Justification 

Connectivity: 

Things  in  the  environment 
influence  one  another  in 
complicated  and 
unpredictable  ways 

Examples  of  decisions  which  involved  interrelated  factors 
included: 

All  groups  were  requesting  information  from  the  DM. 

Many  requests  for  information  were  for  information  that 
would  have  been  outside  of  the  CF’s  lawful  mandate. 

There  were  requests  from  local  security  partners  for 
information  about  CF  capacity  and  capabilities  but  the  DM 
could  not  tell  them  (secret  or  operationally  sensitive 
information).  This  situation  required  a  lot  of  juggling  to 
figure  out  what  the  DM  can  do,  what  the  DM  can  say,  how 
the  DM  can  get  around  those  situations  and  make  it  work 
without  stepping  outside  of  CF  bounds. 

The  breakdown  of  C2  during  the  event  had  profound 
effects.  For  example,  federal  authorities  decided  that  they 
were  going  to  take  over  from  local  authorities.  This  had 
many  implications  for  the  DM  including  the  fact  that 
tactical  officers  had  to  be  moved  from  one  location  to 
another  on  short  notice.  However,  the  tactical  officers 
were  not  available  to  redeploy,  which  meant  other  options 
had  to  be  found.  As  well,  if  other  tactical  officers  were  to 
be  brought  in  suddenly,  other  arrangements  have  to  be 
made  including  feeding,  housing,  and  pay.  There  are 
implications  for  C2,  including  whether  the  local  authorities 
would  be  told  to  officially  stand  down,  was  the  army  going 
to  deploy?  Were  CF  assets  at  risk  and  should  they  be 
protected?  What  were  the  ROEs  going  to  be  (the  CF  only 
had  lethal  force)?  So  the  possible  political  implications  of 
the  situation  were  profound.  There  were  follow-on  effects 
such  as  career  implications  and  public  perception  (e.g.,  if 
it  was  perceived  that  the  local  authorities  were  ineffective 
this  could  have  far-reaching  implications).  This  could  also 
have  far-reaching  consequences  for  personal 

High 

Many  factors  that  are 
interrelated  and  that  have 
to  be  considered  when 
making  decisions  (e.g., 
the  negotiation  for  the  use 
of  DND  property  involved 
at  least  1 1  factors  that  the 
DM  had  to  consider) 
Examples  of  2nd  and  3rd 
order  effects  (e.g.,  the 
breakdown  of  C2  involved 
at  least  10  additional 
decisions  or  effects, 
including  effects  on 
dozens  of  personal 
relationships  and 
organizational 
relationships) 
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Concept  and  Definition 

Concept  Examples 

Concept  Importance  for 
Scenario  (High,  Medium, 
Low)  and  Justification 

relationships  and  the  relationship  between  the  security 
stakeholders. 

There  was  a  request  from  the  local  authorities  to  use  CF 
property.  The  DM  was  acting  as  a  liaison  to  help  with 
arrangements.  Originally  the  local  authorities  said  that 
access  to  a  piece  of  property  was  all  they  needed,  but 
upon  questioning  by  the  DM  they  also  wanted  access  to 
many  other  related  facilities  and  amenities 

Dynamics: 

The  system  has  aspects 
that  unfold  over  time.  For 
example,  the  environment 
changes  over  time  even 
when  you  do  nothing;  the 
rate  at  which  things  change 
may  be  variable;  there  may 
be  delays  between  actions 
and  effects. 

It  is  important  to  note 
hierarchical  aspects  if 
present  (i.e.,  subsystems 
that  have  their  own 
dynamics  which  are  part  of 
the  DM  context). 

Situations  that  unfolded  over  time  that  profoundly  affected 
decision  making  included: 

Strategic  level  decisions  were  made  at  the  latest  possible 
moment.  This  meant  that  the  rate  at  which  plans  had  to  be 
made  and  implemented  kept  changing,  and  eventually 
tactical  activities,  like  contracting,  were  done  in  haste; 
Treasury  Board  regulations  and  PWGSC  contracting 
guidelines  were  not  well-understood  by  Provincial  and 
Municipal  partners.  There  was  general  ignorance  about 
the  best  way  to  negotiate  for  the  use  of  federal  assets  for 
security  related  to  the  event.  Originally  local  security 
partners  were  negotiating  for  federal  resources,  but  they 
were  unprepared  when  the  contracting  process  took  a  lot 
of  time  and  they  received  massive  bills  for  the  use  of 
these  assets.  The  DM  suggested  that  the  process  should 
be  changed  so  that  requests  were  made  from  federal 
rather  than  local  security  partners.  When  this  was  done, 
the  cost  of  many  security  aspects  (e.g.,  renting  a  CF 
armoury)  went  down  to  nothing  and  the  timelines  for 
approval  decreased.  According  to  the  DM,  if  this  strategy 
had  been  done  in  the  beginning  much  difficulty,  wasted 
effort,  and  bad  feelings  could  have  been  avoided; 

Over  time  it  became  clear  to  the  DM  that  vital  information 
was  not  known  (e.g.,  knowledge  of  the  legal  case  Regina 
vs.  Knowlton,  a  case  where  the  finding  resulted  in  the 
authority  to  have  cordons  and  protect  them  when 
countries  are  hosting  internationally  protected  persons). 

The  DM  generally  decided  to  inform  people  about  the 
information  (felt  would  be  remiss  not  to),  but  then  the  DM 
was  marginalized  because  of  this  by  some  people 
because  who  felt  the  DM  was  “sticking  their  nose  in”.  This 
caused  concerns  for  the  DM’s  interpersonal  relationships 
and  also  conflicts  between  others  because  some  people 
would  agree  with  the  DM  and  some  would  disagree,  and 
the  implications  of  these  incidents  built  over  time; 

The  individual  agendas  of  security  personnel  involved 
played  a  role  in  how  events  unfolded.  Certain  individuals 
were  trying  to  keep  information  to  themselves  (which  led 
to  delays  in  needed  information  being  communicated), 
others  appeared  to  have  other  priorities  than  maintaining 

High 

Nine  specific  examples  of 
situations  that  unfolded 
over  time  that  profoundly 
affected  decision  making 

Twenty-seven  subsystems 
with  somewhat  different 
dynamics  which  impacted 
decision  making 
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Concept  Examples 

Concept  Importance  for 
Scenario  (High,  Medium, 
Low)  and  Justification 

security  for  the  event,  etc.; 

Personality  played  a  role  and  the  interactions  between 
personality  and  events  changed  over  time;  There  were 
several  C2  cells  actively  coordinating  during  the  event. 

The  DM  was  present  in  the  LCC  and  the  “main”  HQ  was 
located  elsewhere,  so  this  created  a  high  demand  on  the 

DM  for  information  as  the  DM  was  the  person  “on  the 
ground”  with  access  to  information  about  events  occurring 
at  the  international  event.  As  events  unfolded,  the 
pressure  became  more  and  more  intense; 

The  demonstrations  and  protests  unfolded  over  time,  as 
well  as  their  implications,  such  as  how  people  were 
handling  the  situation,  how  C2  would  operate,  what  extra 
security  forces  were  available,  etc.; 

The  breakdown  of  C2  during  the  event  unfolded  over  time 
and  had  profound  effects,  including  far-reaching 
consequences  for  personal  relationships  and  the 
relationship  between  security  stakeholders; 

Investigations  are  on-going,  and  these  investigations  will 
unfold  over  time  and  have  many  consequences  for 
organizational  and  personal  relationships,  organizational 
reputation,  people’s  careers,  recommendations  for  future 
events,  etc.  As  the  understanding  that  there  would  likely 
be  investigations  related  to  this  event  grew,  this 
influenced  decision-making  and  some  individual’s 
decision  making  became  more  and  more  self-protective. 
There  were  subsystems  which  had  their  own  dynamics  in 
this  context.  These  included: 

•  At  least  22  different  security  agencies. 

Dynamics  of  the  security  personnel  were 
different  based  on  location  as  they  had  different 
types  of  areas  to  police,  different  groups, 
different  numbers  of  attendees/observers,  etc.; 

•  Dynamics  of  the  security  personnel  were  also 
different  based  on  organization  as  they  had 
different  planning  approaches  and  other 
processes,  different  cultures,  and  different 
timelines  over  which  they  were  accustomed  to 
working; 

•  Other  subsystems  with  different  dynamics 
(cultures,  timelines,  processes)  included: 

o  Peaceful  protestors; 
o  Violent  protestors; 
o  Attendees  of  the  event; 
o  General  citizens; 

o  Media. 
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Concept  Examples 

Concept  Importance  for 
Scenario  (High,  Medium, 
Low)  and  Justification 

Multiple  conflicting  goals: 
Having  to  achieve  multiple 
objectives  which  may  not 
be  all  achievable  at  the 
same  time 

The  goals  of  the  DM  included: 

•  Building  and  maintaining  personal  relationships; 

•  Building  and  maintaining  relationships  between 
organizations; 

•  Meeting  responsibilities  to  the  situation; 

•  Meeting  obligations  to  higher  command; 

•  Providing  advice  about  CF  capabilities  to 
security  organizations; 

•  Maintaining  the  security  of  confidential  CF 
assets; 

•  Supporting  negotiations  for  the  use  of  federal 
assets  for  security  related  to  the  event; 

•  Making  sure  actions  are  legal; 

•  Making  sure  actions  are  defensible  morally; 

•  Making  sure  the  CF  does  not  “become  the  face” 
of  the  event  (i.e.,  the  CF  wanted  to  advise  and 
stay  behind  the  scenes). 

Most  of  these  goals  can  conflict  depending  on  the 
situation;  for  example,  building  and  maintaining 
relationships  with  one  group  was  sometimes  seen  as 
negative  by  another  group  and  so  hurt  the  relationship 
with  them;  providing  certain  information  could  damage 
relationships  if  it  was  seen  negatively  (e.g.,  it  was  seen  as 
“sticking  your  nose  in”);  providing  information  to  local 
security  personnel  often  contradicted  the  goal  of 
maintaining  the  security  of  confidential  CF  assets. 

High 

There  were  an  extremely 
high  number  of  important 
goals  that  conflicted  in 
different  ways  in  different 
situations  (10  distinct 
goals) 

Three  examples  of  goal 
conflict 

Under-specified  goals: 

Goals  may  be  difficult  to 
achieve  because  they  are 
too  vague 

Examples  of  underspecified  goals  included: 

The  DM  didn’t  really  get  clear  terms  of  reference  for  the 
job,  which  gave  the  DM  needed  flexibility  but  the  DM’s  job 
might  have  been  easier  with  clearer  terms  of  reference; 
When  asked  what  information  would  be  useful,  the  DM 
was  instructed  by  the  boss  to  “Give  me  everything”,  so 
basically  every  decision  and  conversation  was 
communicated.  There  was  no  other  specific  direction  and 
the  DM  didn’t  actually  know  what  the  boss  needed  to 
know  and  what  was  going  on  at  his  end; 

When  things  started  going  wrong,  the  instruction  was 
generally  “sort  it  out”,  which  was  not  helpful  (although  this 
was  instruction  to  others  more  than  the  DM). 

High 

Three  examples  of  vague 
goals  that  impacted  the 
ability  of  the  DM  to  make 
decisions 

Independent  agents: 

There  are  independent 
entities  in  the  environment 
who  influence  it  (they  may 

Independent  agents  who  influenced  decision  making 
included: 

•  At  least  22  public  order  units  involved  -  there 
were  several  ways  the  DM  mentioned  that  their 

High 

Twenty-eight  independent 
agents  who  impacted 
decision  making 
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Concept  Examples 

Concept  Importance  for 
Scenario  (High,  Medium, 
Low)  and  Justification 

have  different  goals  than 
the  decision  maker) 

goals  conflicted  with  the  DM’s  own.  Because  of 
the  importance  of  the  event,  many  were  in  self¬ 
protection  mode,  which  meant  they  were  not 
cooperative  with  the  DM  or  each  other,  and  also 
were  not  focussing  on  security.  Some  public 
order  units  just  left,  meaning  that  they  were  no 
longer  contributing  to  security.  There  were  many 
difficulties  because  of  territoriality  of  different 
groups  and  the  goal  of  self-protection  held  by 
many  of  them; 

•  Media  with  its  own  biases  and  agenda  (e.g., 
getting  a  good  story); 

•  Mayor  -  wanted  a  briefing  -  caused 
reprioritization  and  pulled  the  DM  away  from 
other  tasks; 

•  Peaceful  protestors; 

•  Violent  protestors  (wanted  to  disrupt  the  event 
and  threatened  security); 

•  Attendees  of  the  event; 

•  General  citizens  (didn’t  want  to  have  to  deal 
with  security  measures  such  as  the  fence). 

•  There  were  issues  with  learning  the  same 
language/terminology  because  collaboration 
was  made  difficult  by  a  lack  of  common 
language. 

Seven  examples  of 
independent  agents  with 
goals  that  conflicted  or 
could  interfere  with  the 

DM’s  goals 

2.  Experience  to  complexity  factor  mapping:  NSE  Officer, 
Afghanistan 


Concept  and  Definition 

Concept  Examples 

Concept  Importance  for 
Scenario  (High,  Medium, 
Low)  and  Justification 

Connectivity: 

Things  in  the 
environment  influence 
one  another  in 
complicated  and 
unpredictable  ways 

Examples  of  decisions  which  involved  interrelated  factors 
included: 

The  DM  was  responsible  for  meeting  logistics  needs  which 
included  quantity,  storage,  maintenance,  and  location  of 
food,  artillery,  vehicles,  and  other  equipment  using  a  limited 
number  of  personnel  and  vehicles  to  transport  items. 

Because  the  resources  were  dispersed  (decentralized),  this 
meant  that  a  lot  of  juggling  had  to  be  done  to  make  sure 
supplies  were  where  they  were  needed; 

The  relative  importance  of  logistics  considerations  in  the  CF 
is  quite  variable  and  frequently  little  effort  is  directed  at  the 
sustainment  estimate.  Because  more  firepower  was  desired 

High 

Many  factors  that  are 
interrelated  and  that  have  to 
be  considered  when  making 
decisions  (e.g.,  the  basic 
logistics  problem  involved 
over  a  dozen  factors 
including  many  different 
vehicles,  many  different 
types  of  items,  many 
different  locations,  and 
limited  staff.  These  were  all 
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Low)  and  Justification 

in  the  Afghanistan  mission  but  the  budget  was  limited,  there 
was  an  increase  in  the  proportion  of  combat  to  logistics 
personnel.  This  meant  that  there  were  a  relatively  small 
number  of  logistics  personnel  sent  to  support  the  combat 
troops,  which  had  both  immediate  and  long-term  effects  on 
the  ability  of  the  logistics  staff  to  keep  up  with  resource 
needs  (e.g.,  impacted  psychological  well-being  and  safety 
of  logistics  staff  due  to  lack  of  sleep  and  rest,  impacted 
number  of  convoys  which  could  occur,  impacted  the  rate  at 
which  items  could  move,  impacted  the  ability  of  the  staff  to 
do  maintenance); 

Another  impact  of  little  effort  being  directed  at  the 
sustainment  estimate  is  that  during  training,  true 
sustainment  was  not  practiced  (e.g.,  soldiers  could  go  to  the 
base  and  get  food  easily).  This  led  to  a  lack  of  information 
for  planners  about  true  logistical  needs  as  well  as  a  lack  of 
training  for  soldiers  regarding  true  sustainment  demands 
and  readiness  requirements,  which  likely  impacted  their 
morale  and  preparedness; 

The  location  of  the  conflict  (Afghanistan)  had  a  profound 
impact  on  logistics.  Afghanistan  is  surrounded  by  other 
countries  (i.e.,  no  sea  access);  thus,  ammunition  has  to  be 
flown  in.  Because  of  the  nature  of  artillery,  flying  artillery 
over  other  countries  requires  their  consent,  and  therefore 
much  preplanning  is  required  to  move  ammunition  (e.g.,  it 
can  take  over  a  month  to  receive  ammunition  from  Canada). 
This  in  turn  results  in  a  lack  of  flexibility  in  the  ammunition 
ordering  process  and  a  resulting  vulnerability  in  the  flow  of 
resources. 

factors  that  the  DM  had  to 
consider) 

Examples  of  2nd  and  3rd 
order  effects  (e.g.,  side 
effects  of  directing  little  effort 
at  the  sustainment  estimate 
involved  at  least  13 
additional  decisions  or 
effects) 

Dynamics: 

The  system  has  aspects 
that  unfold  over  time.  For 
example,  the 
environment  changes 
over  time  even  when  you 
do  nothing;  the  rate  at 
which  things  change  may 
be  variable;  there  may  be 
delays  between  actions 
and  effects. 

It  is  important  to  note 
hierarchical  aspects  if 
present  (i.e.,  subsystems 
that  have  their  own 
dynamics  which  are  part 
of  the  DM  context). 

Situations  that  unfolded  over  time  that  profoundly  affected 
decision  making  included: 

The  enemy  is  constantly  evolving  their  tactics  (e.g.,  their 
ambush  locations  and  methods).  This  means  that  the 
location  of  resources  (e.g.,  FOBs)  became  more  or  less 
relevant  depending  on  enemy  locations  and  activity; 

Locations  and  number  of  FOBs  strongly  influenced  logistics 
but  they  were  planned  without  logistics  input  and  had  to  be 
built  well  before  they  were  needed;  there  was  no  way  to 
influence  where  they  were  or  know  that  they  would  be 
appropriately  situated.  As  events  unfolded,  the  FOBs  were 
not  useful  as  the  OF  were  sent  to  another  part  of 

Afghanistan  to  fight; 

The  relative  importance  of  logistics  considerations  in  the  OF 
is  quite  variable  and  frequently  little  effort  is  directed  at  the 
sustainment  estimate.  Because  more  firepower  was  desired 
in  the  Afghanistan  mission  but  the  budget  was  limited,  there 
was  an  increase  in  the  proportion  of  combat  to  logistics 
personnel.  This  meant  that  there  were  relatively  few 

High 

Eight  specific  examples  of 
situations  that  unfolded  over 
time  that  profoundly  affected 
decision  making 

At  least  twelve  subsystems 
with  somewhat  different 
dynamics  which  impacted 
decision  making 
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logistics  personnel  sent  to  support  the  combat  troops,  which 
had  both  immediate  and  long-term  effects  on  the  ability  of 
the  logistics  staff  to  keep  up  with  resource  needs  (e.g., 
impacted  psychological  well-being  and  safety  of  logistics 
staff  due  to  lack  of  sleep  and  rest,  impacted  number  of 
convoys  which  could  occur,  impacted  the  rate  at  which 
items  could  move,  impacted  the  ability  of  the  staff  to  do 
maintenance); 

Another  impact  of  directing  little  effort  at  the  sustainment 
estimate  was  that  during  training,  true  sustainment  was  not 
practiced  (e.g.,  soldiers  could  go  to  the  base  and  get  food 
easily).  This  led  to  a  lack  of  advanced  information  for 
planners  about  true  logistical  needs  (information  had  to  be 
gathered  over  time  during  the  operation)  as  well  as  a  lack  of 
training  for  soldiers  regarding  true  sustainment  demands 
and  readiness  requirements,  which  likely  impacted  their 
morale  and  preparedness; 

The  location  of  the  conflict  (Afghanistan)  had  a  profound 
impact  on  logistics.  Afghanistan  is  surrounded  by  other 
countries  (i.e.,  no  sea  access);  thus,  ammunition  has  to  be 
flown  in.  Because  of  the  nature  of  artillery,  flying  artillery 
over  other  countries  requires  their  consent,  and  therefore 
much  preplanning  is  required  to  move  ammunition  (e.g.,  it 
can  take  over  a  month  to  receive  ammunition  from  Canada). 
This  in  turn  results  in  a  lack  of  flexibility  in  the  ammunition 
ordering  process  and  a  resulting  vulnerability  in  the  flow  of 
resources; 

There  was  a  radical  change  in  the  CONOPS  for  the 
Afghanistan  mission  which  occurred  immediately  prior  to  the 
DM  landing  in  Afghanistan.  This  CONOPS  required 
changes  in  plans  to  how  the  LAV  companies  would  be  used 
(i.e.,  LAV  companies  were  always  in  use  rather  than  having 
one  on  base).  This  change  in  resource  allocation  had  long¬ 
term  repercussions  to  how  maintenance  could  be  scheduled 
and  performed  (e.g.,  all  LAVs  constantly  being  out  in  use 
means  no  preventative  maintenance  was  possible); 

The  use  of  ammunition  varied  greatly  over  time.  Monitoring 
ammunition  use  at  the  beginning  of  the  DM’s  tour  indicated 
that  X  amount  was  used  per  unit  time;  however,  this 
increased  greatly  when  the  pace  of  battle  increased  as  well 
as  when  other  factors  changed  (commanders  changed  and 
some  commanders  were  more  prone  to  use  a  lot  of 
ammunition).  Factors  which  increased  the  use  of  resources 
(especially  ammunition)  also  had  other  effects  such  as 
making  it  more  difficult  for  the  BG  to  report  ammunition  use 
(i.e.,  as  the  pace  of  battle  increased,  the  use  of  ammunition 
greatly  increased,  but  the  information  from  the  BG  about 
how  much  artillery  was  being  used  was  not  sent;  this  could 
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have  been  due  to  the  stress  and  time  pressure  of  battle  or 
other  factors); 

Resource  availability  was  so  close  to  the  minimum  required 
that  it  had  to  constantly  be  balanced  and  plans  changed; 
factors  such  as  vehicles  needing  repair  would  mean  that 
rebalancing  was  required  frequently. 

There  were  subsystems  which  had  their  own  dynamics  in 
this  context.  These  included: 

•  The  Battle  Group  (BG); 

•  The  logistics  unit; 

•  Personnel  at  the  FOBs  (multiple  groups); 

•  The  CF  as  a  whole; 

•  CEFCOM; 

•  CANOSCOM; 

•  Other  armies  (e.g.,  US,  British,  Netherlands); 

•  The  ANA 

•  The  Taliban  (the  enemy) 

•  For  example,  the  events  experienced  by  the  BG 
have  profound  effects  on  logistics,  but  these 
effects  were  sometimes  delayed. 

•  Personnel  manning  different  FOBs  experienced 
different  levels  of  activity  and  required  different 
logistical  support. 

•  Enemy  activity  also  has  profound  effect  on 
logistics  requirements  and  it  waxes  and  wanes 
unpredictably. 

Multiple  conflicting  goals: 
Having  to  achieve 
multiple  objectives  which 
may  not  be  all  achievable 
at  the  same  time 

The  goals  of  the  DM  included: 

•  Maintaining  Canadian  security 

•  Making  sure  that  Canada  was  well  represented 

•  Taking  away  Taliban  training  areas 

•  Maintaining  a  positive  public  perception  of  the  CF 

•  Making  and  keeping  strong  relationships  with 
allies 

•  Lowering  the  stress  level  of  commanding  officers 

•  Managing  physical  resources  (food,  artillery, 
vehicles,  other  equipment) 

•  Acting  within  resource  constraints. 

The  DM  indicated  that  many  of  these  goals  were 
extremely  important  and  yet  had  to  be  delicately 
balanced  due  to  lack  of  resources.  For  example,  the 
need  to  share  resources  with  allies  for  tactical  effect 
can  sometimes  compromise  resupply  of  other 

High 

There  were  an  extremely 
high  number  of  important 
goals  that  conflicted  in 
different  ways  in  different 
situations  (8  distinct  goals) 
Three  examples  of  goal 
conflict 
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Canadian  units;  time  lags  in  resupply  meant  that  the 
balance  between  managing  physical  resources  (i.e., 
having  needed  resources)  and  acting  within  resource 
constraints  (i.e.,  not  spending  too  much  money  and 
having  extra  supplies  not  being  used)  were  in  conflict. 

Under-specified  goals: 
Goals  may  be  difficult  to 
achieve  because  they  are 
too  vague 

Examples  of  underspecified  goals  included: 

•  What  role  the  logistics  component  and  CF  was 
supposed  to  have  with  the  ANA  was  not  fully 
developed  when  the  DM  was  involved.  The  DM 
was  just  told  to  “do  what  you  can”  to  work  with 

ANA  logistics. 

•  The  DM  noted  that,  although  difficult  to  achieve, 
goals  were  generally  clear  (at  least  at  a  high 
level). 

Medium 

One  example  of  a  vague 
goal  that  impacted  the  ability 
of  the  DM  to  make  decisions 

Independent  agents: 

There  are  independent 
entities  in  the 
environment  who 
influence  it  (they  may 
have  different  goals  than 
the  decision  maker) 

Independent  agents  who  influenced  decision  making 
included: 

•  The  CF  (ensure  Canadian  security  is  preserved, 
make  sure  Canada  is  well  represented,  take  away 
Taliban  training  areas); 

•  US  army  (make  sure  US  is  well  represented); 

•  NATO  (follow  NATO  doctrine  and  ensure  it  is 
followed  by  others); 

•  Netherlands  army  (make  sure  Netherlands  are 
well  represented); 

•  British  Army  (make  sure  British  are  well 
represented); 

•  ROMP  (mentor  the  ANP); 

•  Cl  DA  (provide  humanitarian  aid); 

•  DFAIT  (assist  in  provincial  reconstruction 
program); 

•  ANA  (protect  own  country,  maintain 
independence); 

•  Taliban  (defeat  enemies  including  CF,  other 

NATO  forces). 

Many  independent  agents  with  conflicting  resource  needs 
which  had  to  be  accommodated  (e.g.,  CF,  other  allied 
forces,  ANA,  NATO  personnel,  US  army) 

Several  groups  (e.g.,  Taliban,  ANA  at  times)  had  goals 
which  were  in  conflict  with  the  DM’s  goals. 

High 

Ten  independent  agents 
who  impacted  decision 
making 

Seven  examples  of 
independent  agents  with 
goals  that  conflicted  or  could 
interfere  with  the  DM’s  goals 
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Connectivity: 

Things  in  the 
environment 
influence  one 
another  in 
complicated  and 
unpredictable 
ways 

Examples  of  decisions  which  involved  interrelated  factors  included: 

The  prison  break  led  to  a  process  of  information  gathering.  This 
process  gradually  revealed  that  there  was  no  Common  Operating 
Picture  (COP)  among  Afghan  security  agencies;  that  the  ANP  could 
not  effectively  get  information  about  threats;  that  the  ANP  had 
difficulty  responding  to  threats;  that  the  ANP  could  not  effectively  ask 
for  assistance  from  other  security  organizations;  and  that  Afghan 
security  organizations  typically  work  at  a  tactical  level  and  are  not 
used  to  working  at  an  operational  or  strategic  level.  These  pieces  of 
information  changed  how  the  DM  saw  his  goals  and  how  they  could 
be  accomplished; 

Cultural  differences  between  Afghanistan  and  Canada  are  profound; 
they  created  a  lack  of  trust  and  difficulty  with  coordination,  affecting 
many  of  the  DM’s  decisions.  Factors  included: 

•  Extremely  high  emphasis  on  interpersonal  relationships  in 
Afghanistan  (e.g.,  the  importance  of  personal  relationships 
to  Afghans  is  paramount).  This  has  profound  implications; 
for  example,  authority  based  on  position  alone  is  not  “real” 
authority  to  the  Afghans  (obedience  is  based  on  personal 
relationships); 

•  Differences  in  the  legal  system  and  widespread  corruption 
(e.g.,  people  responsible  for  the  prison  break  probably 
were  not  punished  as  they  could  pay  to  be  released,  which 
destroyed  trust); 

•  A  difference  in  the  idea  of  what  it  is  to  have  a  job  and  the 
attendant  responsibility  (e.g.,  a  police  officer  threw  away 
his  phone  because  DM  was  getting  too  many  91 1  calls, 
even  though  that  was  the  only  phone  the  calls  were 
coming  to); 

•  The  Afghans  appeared  to  have  the  perspective  that  they 
had  little  control  over  events  (e.g.,  “what  Allah  wills  will 
happen”);  whereas  Canadians  typically  feel  they  do  have 
control  over  events.  This  had  implications;  for  example,  the 
Afghans  were  not  used  to  the  concept  of  practicing  for  a 
possible  event  and  were  resistant  to  such  training 

•  Different  social  standards  (e.g.,  DM  never  met  the  wife  of 
any  Afghan  the  DM  worked  with).  The  DM  knew  that  social 
relationships  were  important,  but  was  not  able  to  actually 
understand  what  the  relationships  were  as  he  was  not 
privy  to  how  families  were  interrelated  through  marriage; 

•  Literacy  is  so  low  in  Afghanistan  that  usually  no  written 
records  are  kept,  which  means  data  is  not  available  that 
would  help  assess  resource  and  training  requirements; 

High 

Many  factors  that  are 
interrelated  and  that  have 
to  be  considered  when 
making  decisions  (e.g., 
there  were  15  important 
cultural  effects  and 
implications  which 
influenced  the  DM’s 
decisions) 

Examples  of  2nd  and  3rd 
order  effects  (e.g.,  the  DM 
having  to  work  in  a  different 
culture  involved  at  least  15 
additional  decisions  or 
effects) 
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•  Lack  of  appreciation  for  resources  (e.g.,  communications 
equipment  was  given  to  the  relevant  stakeholders;  one 
faction  of  the  ANA  changed  locations  but  left  the 
equipment  in  the  old  location  unattended  without  notifying 
the  DM); 

•  Age  is  more  of  a  factor  in  creating  a  sense  of  authority  in 
Afghanistan.  The  DM  felt  that  more  respect  would  have 
been  received  if  DM  had  been  older; 

Dynamics: 

The  system  has 
aspects  that 
unfold  over  time. 

For  example,  the 
environment 
changes  over 
time  even  when 
you  do  nothing; 
the  rate  at  which 
things  change 
may  be  variable; 
there  may  be 
delays  between 
actions  and 
effects. 

It  is  important  to 
note  hierarchical 
aspects  if  present 
(i.e.,  subsystems 
that  have  their 
own  dynamics 
which  are  part  of 
the  DM  context). 

Situations  that  unfolded  overtime  that  profoundly  affected  decision 
making  included: 

There  was  a  91 1  system  that  was  implemented;  this  was  supposed 
to  be  run  by  the  ANP  but  the  chief  of  police  discarded  the  phone  that 
was  used  for  the  91 1  calls  as  he  found  it  inconvenient.  The  DM 
volunteered  to  take  over  the  91 1  function;  this  became  a  tool  to  get 
the  ANP  involved  as  the  operations  room  would  receive  the  calls  but 
the  ANP  would  be  contacted  to  answer  them.  This  created  additional 
(an  unpredictable)  windows  of  opportunity  for  collaboration  with  the 
ANP,  for  obtaining  intelligence,  and  for  facilitating  relationships  with 
civilians; 

The  prison  break  led  to  a  process  of  information  gathering.  This 
process  gradually  revealed  that  there  was  no  Common  Operating 
Picture  (COP)  among  Afghan  security  agencies;  that  the  ANP  could 
not  effectively  get  information  about  threats;  that  the  ANP  had 
difficulty  responding  to  threats;  that  the  ANP  could  not  effectively  ask 
for  assistance  from  other  security  organizations;  and  that  Afghan 
security  organizations  typically  work  at  a  tactical  level  and  are  not 
used  to  working  at  an  operational  or  strategic  level.  These  pieces  of 
information  changed  how  the  DM  saw  his  goals  and  how  they  could 
be  accomplished; 

The  DM  attempted  to  get  the  relevant  stakeholders  to  install, 
maintain,  train  on,  and  use  communication  equipment  provided  by 
the  U.S.  DM  found  this  very  difficult  for  many  reasons  (e.g.,  cultural 
differences),  and  felt  that  any  intervention  tried  did  not  result  in 
change  (i.e.,  there  seemed  to  be  a  “set  point”  in  the  environment  that 
was  highly  resistant  to  change).  This  became  more  and  more 
frustrating  as  time  went  on  and  the  DM  grew  increasingly  hopeless 
that  the  C2  infrastructure  would  be  adequate  to  maintain  security. 

There  were  subsystems  which  had  their  own  dynamics  in  this 
context.  These  included: 

•  The  ANA; 

•  The  ANP 

•  Kandahar  prison  personnel; 

•  The  CF; 

•  Civilians; 

•  The  operations  centre  in  which  the  DM  was  working. 

Medium 

Three  specific  examples  of 
situations  that  unfolded 
over  time  that  profoundly 
affected  decision  making 

Six  subsystems  with 
somewhat  different 
dynamics  which  impacted 
decision  making 
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Canadian  and  other  foreign  military  organizations  appeared  to  have 
a  relatively  fast  rate  of  change  compared  to  the  Afghan 
organizations  (e.g.,  longer  time  to  train,  longer  time  to  change 
procedures),  with  an  exception  being  in  the  rate  of  personnel 
change,  with  higher  turnover  in  the  Afghan  organizations. 

Multiple 

conflicting  goals: 
Having  to  achieve 
multiple 

objectives  which 
may  not  be  all 
achievable  at  the 
same  time 

The  goals  of  the  DM  included: 

•  Creating  an  effective  Kandahar  city  security  network 

•  Building  and  maintaining  relationships  with  members  of  the 
ANA,  ANP,  Kandahar  prison  personnel,  Border  Security, 
and  the  NDS. 

There  were  some  cases  in  which  it  was  difficult  to  achieve  all  of 
these  goals.  One  example  was  that  the  ANP  and  ANA  were 
adversaries  that  the  DM  was  trying  to  get  to  work  together,  and  it 
was  difficult  to  meet  both  of  their  needs  and  expectations.  Another 
example  is  that  the  DM  had  to  balance  the  feedback  accuracy  that 
the  DM  provided  to  trainees  with  allowing  them  to  save  face  and 
remain  invested  (i.e.,  too  much  negative  feedback  would  likely  have 
resulted  in  stakeholders  withdrawing  from  the  training). 

Medium 

There  were  a  number  of 
important  goals  that 
conflicted  in  different  ways 
in  different  situations  (6 
distinct  goals) 

Two  examples  of  goal 
conflict 

Primary  difficulty  due  to 
problems  with  getting 
stakeholder  buy-in  and 
participation  rather  than 
goals  conflicting 

Under-specified 

goals: 

Goals  may  be 
difficult  to  achieve 
because  they  are 
too  vague 

Examples  of  underspecified  goals  included: 

The  goal  of  creating  an  effective  Kandahar  city  security  network  was 
superficially  clear  but  what  was  actually  required  and  the  poor  state 
of  the  current  system  was  only  revealed  over  time. 

Medium 

One  example  of  a  vague 
goal  that  impacted  the 
ability  of  the  DM  to  make 
decisions 

Goals  at  a  high  level  were 
fairly  clear,  the  main 
challenge  was  to  determine 
current  security  situation 
and  needs 

Independent 

agents: 

There  are 
independent 
entities  in  the 
environment  who 
influence  it  (they 
may  have 
different  goals 
than  the  decision 
maker) 

Independent  agents  who  influenced  decision  making  included: 

•  The  CF; 

•  The  ANA  (didn’t  like  working  with  the  ANP  and  vice  versa); 

•  The  ANP; 

•  Kandahar  prison  personnel  (not  trusted;  most  were  new 
after  the  prison  break); 

•  Border  security; 

•  NDS; 

•  IEC  (Independent  Electoral  Commission)  -  required 
protection; 

•  Mentors. 

Large  problems  due  to  interpersonal  conflicts  between  different 
stakeholders  (e.g.,  ANA  and  ANP  didn’t  want  to  work  together;  no 
one  wanted  to  work  with  the  Kandahar  prison  personnel) 

High 

Eight  independent  agents 
who  impacted  decision 
making 

Seven  examples  of 
independent  agents  with 
goals  that  conflicted  or 
could  interfere  with  the 

DM’s  goals 
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Concept  Examples 

Concept  Importance  for 
Scenario  (High,  Medium, 
Low)  and  Justification 

Large  problems  related  to  getting  stakeholder  buy-in  and  adequate 
participation;  this  partially  a  cultural  problem  (e.g.,  all  Afghani  groups 
didn’t  want  to  cooperate  with  the  DM’s  training  plans) 

4.  Experience  to  complexity  factor  mapping:  CoE  Training 
Developer 


Concept  and  Definition 

Concept  Examples 

Concept  Importance  for 
Scenario  (High,  Medium, 
Low)  and  Justification 

Connectivity: 

Things  in  the  environment 
influence  one  another  in 
complicated  and 
unpredictable  ways 

Examples  of  decisions  which  involved  interrelated  factors 
included:  cultural  information,  working  with  translators, 
team  training,  interacting  with  the  media  (e.g.,  giving 
interviews),  working  with  multiple  OGDs  (e.g.,  DFAIT, 

Cl  DA),  dealing  with  unexpected  events,  and  how  to 
balance  conflicting  objectives  (e.g.,  when  CEFCOM  and 
NATO  have  different  policies  or  priorities) 

Factors  the  DM  must  consider  how  to  include  in  training: 

The  degree  to  which  the  CF  would  have  to  work  with 

Cl  DA,  DFAIT,  and  other  organizations  only  became  clear 
over  time  (e.g.,  the  Manley  panel  report  indicated  a 
greater  need  for  cooperation),  therefore  their  involvement 
in  providing  and  receiving  training  evolved  over  time. 

There  were  several  components  to  this,  including: 

•  It  took  time  to  build  the  necessary  trust  between 
agencies  (e.g.,  CF,  CIDA,  DFAIT)  to  effectively 
train  together.  Networking  and  relationship 
building  were  more  actively  encouraged  over 
time  (e.g.,  a  “night  out  dinner”  was  built  into 

OGD  training  to  facilitate  networking  etc.); 

•  Another  aspect  of  OGD  integration  which  took 
place  over  time  was  getting  a  common 
understanding  of  basic  Operational  Planning 
Process  (OPP).  Initially  CIDA,  DFAIT  etc.  have 
no  training  in  the  OPP  and  this  basic  training 
was  begun.  This  training  helped  improve 
common  understanding  and  build  social 
networks  which  were  useful  for  training  and  also 
in  theatre.  The  DM  said  that  there  were  many 

2nd  and  3rd  order  effects  from  these  social 
relationships  and  other  training  benefits; 

As  operational  environments  and  priorities  changed, 
training  had  to  change  along  with  them.  This  requires 
almost  constantly  updating  SA  about  operational 
environments  and  priorities  as  well  as  implementing 
changes  to  effectively  deal  with  changes.  Someone  from 

Medium 

Factors  that  are  interrelated 
and  that  have  to  be 
considered  when  making 
decisions  (e.g.,  the  trainers 
have  to  consider  at  least  7 
different  aspects  of  training 
when  designing  the 
programs.) 

Examples  of  2nd  and  3rd 
order  effects  (e.g., 
involvement  of  CIDA  and 
DFAIT  in  the  OPP  training 
involved  “many”  2nd  and  3rd 
order  effects  (although  these 
were  not  explicitly  listed) 
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Concept  and  Definition 

Concept  Examples 

Concept  Importance  for 
Scenario  (High,  Medium, 
Low)  and  Justification 

the  CoE  often  went  on  the  recce  with  the  new  HQ  so  they 
knew  what’s  needed  in  theatre  and  what  the  current 
situation  was. 

Dynamics: 

The  system  has  aspects 
that  unfold  over  time.  For 
example,  the  environment 
changes  over  time  even 
when  you  do  nothing;  the 
rate  at  which  things  change 
may  be  variable;  there  may 
be  delays  between  actions 
and  effects. 

It  is  important  to  note 
hierarchical  aspects  if 
present  (i.e.,  subsystems 
that  have  their  own 
dynamics  which  are  part  of 
the  DM  context). 

Situations  that  unfolded  over  time  that  profoundly  affected 
decision  making  included: 

As  operational  environments  and  priorities  changed, 
training  had  to  change  along  with  them.  This  required 
almost  constantly  updating  SA  about  operational 
environments  and  priorities  as  well  as  implementing 
changes  to  effectively  deal  with  changes.  Someone  from 
the  CoE  often  went  on  the  recce  with  the  new  HQ  so  they 
knew  what’s  needed  in  theatre  and  what  the  current 
situation  was; 

The  degree  to  which  the  CF  would  have  to  work  with 

Cl  DA,  DFAIT,  and  other  organizations  only  became  clear 
over  time  (e.g.,  the  Manley  panel  report  indicated  a 
greater  need  for  cooperation),  therefore  their  involvement 
in  providing  and  receiving  training  evolved  over  time. 

There  were  several  components  to  this,  including: 

•  It  took  time  to  build  the  necessary  trust  between 
agencies  (e.g.,  CF,  CIDA,  DFAIT)  to  effectively 
train  together.  Networking  and  relationship 
building  were  more  actively  encouraged  over 
time  (e.g.,  a  “night  out  dinner”  was  built  into 

OGD  training  to  facilitate  networking  etc.); 

•  Another  aspect  of  OGD  integration  which  took 
place  over  time  was  getting  a  common 
understanding  of  basic  Operational  Planning 
Process  (OPP).  Initially  CIDA,  DFAIT  etc.  have 
no  training  in  the  OPP  and  this  basic  training 
was  begun.  This  training  helped  improve 
common  understanding  and  build  social 
networks  which  were  useful  for  training  and  also 
in  theatre.  The  DM  said  that  there  were  many 

2nd  and  3rd  order  effects  from  these  social 
relationships  and  other  training  benefits; 

The  money  available  for  training  decreased  over  time  -  at 
first,  when  war  started,  the  budget  was  virtually  unlimited, 
but  funding  became  more  scarce  as  time  when  on  and  the 
mission  began  to  wind  up,  the  recession  started,  etc.; 

Generally  training  programs  use  rolling-wave  planning  as 
there  are  many  variables  which  have  to  be  set  and  things 
change  over  time  (e.g.,  people’s  availability,  what  training 
needs  are); 

Networks  of  people  were  built  as  the  training  was 
developed,  and  this  helped  in  the  acquisition  of  resources 
for  training  as  well  as  getting  other  organizations  involved. 

Medium 

Eight  specific  examples  of 
situations  that  unfolded  over 
time  that  profoundly  affected 
decision  making 

Six  subsystems  with 
somewhat  different 
dynamics  which  impacted 
decision  making 
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Scenario  (High,  Medium, 
Low)  and  Justification 

One  or  two  people  would  get  on  board  (e.g.,  one  member 
of  DFAIT)  and  then  their  contacts  were  useful  forgetting 
more  people  involved.  Personal  contacts  often  proved 
much  more  useful  than  “official”  channels; 

Now,  trainers  are  often  trainees  from  an  earlier  group. 

This  means  that  they  generally  understand  what  they 
need  to  talk  about  and  can  improve  training; 

Changes  to  training  programs  have  a  distinct  pattern. 
Changes  are  difficult  to  make  within  a  training  cycle,  but 
there  is  a  great  deal  of  change  that  happens  between 
training  cycles. 

There  were  subsystems  which  had  their  own  dynamics  in 
this  context.  These  included: 

•  OGDs  (DFAIT,  CIDA) 

•  Trainees 

•  Instructors 

•  Writing  group  (SMEs) 

•  CF 

•  CoE  military  vs.  civilian  chain  of  command 

•  CIDA  and  DFAIT  have  different  deployment 
schedules  than  the  CF  (CF  HQ  9  months,  OGDs 

6  months)  which  causes  problems  with  building 
networks  and  understanding  capabilities  of 
individuals. 

•  CoE  military  vs.  civilian  chain  of  command  - 
both  chains  of  command  have  different  authority 
structures  and  processes. 

Multiple  conflicting  goals: 

Having  to  achieve  multiple 
objectives  which  may  not 
be  all  achievable  at  the 
same  time 

The  goals  of  the  DM  included: 

•  Getting  the  trainees  ready  for  war  (main  goal)  - 
this  is  really  clear.  Generally  this  goal  comes 
with  formation  battle  task  standards  (a  list  of 
things  the  HQ  needs  to  be  able  to  do  in  war); 

•  Coordinating  with  operational  needs  of  trainees; 

•  Meeting  requirements  of  superiors; 

•  Building  and  maintaining  network  of  contacts. 
Sometimes  the  training  goals  of  the  government  and 
higher  HQ  don’t  align 

Training  needs  are  sometimes  in  conflict  with  operational 
needs  (e.g.,  time  for  training  vs.  time  for  performing 
operational  tasks) 

Some  conflicts  arise  simply  due  to  time  limitations  and 
other  resource  limitations  (e.g.,  can’t  do  everything  with 
limited  resources) 

Medium 

There  were  4  important 
goals  that  conflicted  in 
different  ways  in  different 
situations 

Three  examples  of  goal 
conflict 
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Concept  Examples 

Concept  Importance  for 
Scenario  (High,  Medium, 
Low)  and  Justification 

Under-specified  goals: 

Goals  may  be  difficult  to 
achieve  because  they  are 
too  vague 

Examples  of  underspecified  goals  included: 

When  the  training  program  began  initially,  the  DM  was 
working  “in  a  void”,  had  only  limited  staff  and  didn’t  really 
know  what  they  needed; 

The  DM  had  never  experienced  the  kind  of  meeting  that 
they  were  trying  to  recreate,  which  made  it  more  difficult 
to  understand  what  they  were  trying  to  do; 

Some  changes  did  occur  which  were  vague.  For  example, 
the  Canadian  Government  announced  a  counter¬ 
insurgency  strategy  but  failed  to  provide  any  firm 
direction; 

NATO  generally  gives  good  direction  but  when  higher 
level  command  changes  so  can  the  direction  (not  so  much 
vague  goals  as  changing  goals). 

High 

Three  examples  of  vague 
goals  that  impacted  the 
ability  of  the  DM  to  make 
decisions 

Independent  agents: 

There  are  independent 
entities  in  the  environment 
who  influence  it  (they  may 
have  different  goals  than 
the  decision  maker) 

Independent  agents  who  influenced  decision  making 
included: 

•  Trainees  (have  conflict  between  training  needs 
and  operational  needs  (e.g.,  may  have  the 
command  group  take  off  to  do  a  recce  for  the 
time  during  which  a  training  seminar  was 
scheduled)); 

•  NATO  (objectives  may  conflict  with  Canadian 
Government  objectives  and  policies); 

•  CoE  civilian  vs.  OF  military  chain  of  command 
(may  have  different  goals  and  processes); 

•  OGDs  (CIDA,  DFAIT,  etc.)  (have  different  goals, 
processes,  and  timelines). 

Often  difficult  to  coordinate  meetings  and  training  because 
people  have  multiple  priorities  and  commitments. 

Medium 

Six  independent  agents 
who  impacted  decision 
making 

Five  examples  of 
independent  agents  with 
goals  that  conflicted  or  could 
interfere  with  the  DM’s  goals 

5.  Experience  to  complexity  factor  mapping:  Chief  of  Staff, 
Strategic  Advisory  Team,  Afghanistan 


Concept  and  Definition 

Concept  Examples 

Concept  Importance  for 
Scenario  (High,  Medium, 
Low)  and  Justification 

Connectivity: 

Things  in  the  environment 
influence  one  another  in 
complicated  and 
unpredictable  ways 

Examples  of  decisions  which  involved  interrelated  factors 
included: 

One  goal  of  the  DM  was  to  build  relationships;  relationship 
networks  were  built  over  time  and  had  many,  sometimes 
unexpected  benefits.  For  example,  the  DM  made  contact 
with  the  chaplains  at  a  nearby  US  base,  and  these 
chaplains  acted  as  a  resource  when  Canadians  needed 

Medium 

Many  factors  that  are 
interrelated  and  that  have  to 
be  considered  when  making 
decisions  (e.g.,  the 
consideration  of  whether  to 
be  openly  military  involved 
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Concept  Importance  for 
Scenario  (High,  Medium, 
Low)  and  Justification 

counselling  (e.g.,  when  a  co-worker  was  killed).  To 
facilitate  relationship  building,  the  DM’s  group  hosted  an 
open  house  BBQ  every  Friday  night,  and  as  relationships 
built  more  and  more  people  would  attend.  This  created  a 
growing  network  of  connections  between  people  and 
facilitated  many  interactions; 

There  was  an  on-going  balancing  act  between  being 
openly  military  (e.g.,  wearing  uniforms,  driving  military 
vehicles)  and  trying  to  pass  as  civilian.  The  costs  (e.g., 
being  targeted,  legal  trouble  and  safety  issues  if  caught 
with  weapons  out  of  uniform)  and  benefits  (e.g.,  being 
protected  in  more  secure  vehicles,  being  protected  by 
military  law  as  on-duty  military  officers).  This  had  far- 
reaching  consequences,  including  strategic  implications  if 
other  country  members  saw  Canadian  military  entering 
Afghan  military  buildings. 

at  least  8  decisions  and 
possible  consequences  that 
the  DM  had  to  consider) 
Examples  of  2nd  and  3rd 
order  effects  (e.g.,  the 
possible  consequences  of 
being  openly  military  or  not 
involved  at  least  6  possible 
consequences) 

Dynamics: 

The  system  has  aspects 
that  unfold  over  time.  For 
example,  the  environment 
changes  over  time  even 
when  you  do  nothing;  the 
rate  at  which  things  change 
may  be  variable;  there  may 
be  delays  between  actions 
and  effects. 

It  is  important  to  note 
hierarchical  aspects  if 
present  (i.e.,  subsystems 
that  have  their  own 
dynamics  which  are  part  of 
the  DM  context). 

Situations  that  unfolded  over  time  that  profoundly  affected 
decision  making  included: 

One  goal  of  the  DM  was  to  build  relationships;  relationship 
networks  were  built  over  time  and  had  many,  sometimes 
unexpected  benefits.  For  example,  the  DM  made  contact 
with  the  chaplains  at  a  nearby  US  base,  and  these 
chaplains  acted  as  a  resource  when  Canadians  needed 
counselling  (e.g.,  when  a  co-worker  was  killed).  To 
facilitate  relationship  building,  the  DM’s  group  hosted  an 
open  house  BBQ  every  Friday  night,  and  as  relationships 
built  more  and  more  people  would  attend.  This  created  a 
growing  network  of  connections  between  people  and 
facilitated  many  interactions; 

There  was  an  on-going  balancing  act  between  being 
openly  military  (e.g.,  wearing  uniforms,  driving  military 
vehicles)  and  trying  to  pass  as  civilian,  which  were 
influenced  by  factors  such  as  the  current  state  of  tension 
in  the  area.  The  costs  (e.g.,  being  targeted,  legal  trouble 
and  safety  issues  if  caught  with  weapons  out  of  uniform) 
and  benefits  (e.g.,  being  protected  in  more  secure 
vehicles,  being  protected  by  military  law  as  on-duty 
military  officers)  have  to  be  weighed.  This  had  far- 
reaching  consequences,  including  strategic  implications  if 
other  country  members  saw  Canadian  military  entering 
Afghan  military  buildings; 

The  chain  of  command  was  fuzzy.  Task  Force 

Afghanistan  (TFA)  in  Kandahar  was  in  charge  of  the  DM’s 
group  for  administrative  purposes,  but  the  team  actually 
worked  directly  through  CEFCOM.  As  time  went  on  it 
became  more  and  more  clear  that  TFA  felt  they  had  more 
authority  over  the  DM’s  team  than  the  DM  thought,  and 
this  made  decision  making  more  difficult.  For  example, 

Medium 

Five  specific  examples  of 
situations  that  unfolded  over 
time  that  profoundly  affected 
decision  making 

Six  subsystems  with 
somewhat  different 
dynamics  which  impacted 
decision  making 
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TFA  kept  pressuring  the  DM  to  get  electronic  counter¬ 
measures  put  on  their  vehicles,  but  the  DM  kept  putting 
this  off  as  that  would  make  them  targets  (because  it  would 
be  obvious  they  were  military  vehicles); 

Team  cohesion  issues  waxed  and  waned  during  this  time 
period.  For  example,  the  team  member  who  experienced 
the  negligent  discharge  was  teased  about  it  by  the  group 
for  some  time  afterward.  The  group  lived  together  and  had 
to  be  a  tight-knit  group,  so  any  tensions  could  profoundly 
impact  the  mood  of  the  entire  compound  and  the  ability  of 
team  members  to  work  effectively  together.  The  DM 
experienced  2  different  team  configurations  and  the 
second  configuration  were  more  team  players  than  the 
first; 

Events  which  threatened  the  security  of  the  compound  led 
to  reassessment  of  the  security  plan  for  the  compound  as 
well  as  an  on-going  effort  to  work  with  the  British  and 
other  embassies  to  arrange  for  a  more  secure  location  in 
case  an  evacuation  was  required,  and  also  an  attempt  to 
improve  communication  between  these  stakeholders. 

There  were  subsystems  which  had  their  own  dynamics  in 
this  context.  These  included: 

•  TFA; 

•  CEFCOM; 

•  The  DM’s  team; 

•  Forces  from  different  countries; 

•  The  group  at  the  US  base  near  the  DM’s 
compound; 

•  The  Afghan  security  force  hired  by  the  DM. 

The  DM’s  group  had  fewer  resources  than  many  groups 
and  so  were  more  affected  by  change  (e.g.,  need  for 
resource  allocation); 

The  TFA  and  CEFCOM,  as  well  as  other  countries’  forces 
had  different  rates  of  information  exchange  and  slower 
processes  as  they  were  much  larger  groups  than  the 

DM’s  team; 

The  Afghan  security  force  had  a  different  culture  and 
different  practices  than  the  DM’s  team. 

Multiple  conflicting  goals: 

Having  to  achieve  multiple 
objectives  which  may  not 
be  all  achievable  at  the 
same  time 

The  goals  of  the  DM  included: 

•  Ensuring  team  members’  safety 

•  Ensuring  safety  of  the  compound  and 
equipment 

•  Building  relationships 

•  Performing  administration  tasks  effectively 

Medium 

There  were  a  number  of 
important  goals  that 
conflicted  in  different  ways  in 
different  situations  (6  distinct 
goals) 

Two  examples  of  goal 
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•  Dealing  with  unexpected  events  effectively 

•  Meeting  demands  and  requirements  of  higher 
command 

•  Some  of  these  goals  conflicted  -  for  example, 
higher  command  at  TFA  ordered  the  DM  to 
have  the  vehicles  fitted  with  electronic  counter¬ 
measures,  but  the  DM  thought  that  this  would 
make  his  team  targets  and  actually  decrease 
their  safety; 

•  One  main  problem  with  managing  multiple  goals 
was  lack  of  time  rather  than  goals  being 
inherently  contradictory. 

conflict  provided 

Under-specified  goals: 

Goals  may  be  difficult  to 
achieve  because  they  are 
too  vague 

Examples  of  underspecified  goals  included: 

•  There  was  the  requirement  to  assist  the 

Afghans  and  “do  it  their  way”,  but  what  this 
actually  meant  was  unclear. 

Medium 

One  example  of  vague  goals 
that  impacted  the  ability  of 
the  DM  to  make  decisions 

Independent  agents: 

There  are  independent 
entities  in  the  environment 
who  influence  it  (they  may 
have  different  goals  than 
the  decision  maker) 

Independent  agents  who  influenced  decision  making 
included: 

•  DM’s  team  members 

•  American  coalition  groups 

•  ISAF  mission  groups 

•  UN  mission  groups 

•  UN  agencies 

•  Other  nations  with  embassies  and  ambassadors 

•  Afghan  civilians 

•  Afghan  military 

•  Enemy 

It  was  clear  that  the  DM  thought  these  groups  had 
different  agendas  but  clear  examples  were  not  provided; 

Potential  enemy  action  drove  many  decisions. 

Medium 

Nine  independent  agents 
who  impacted  decision 
making 

One  explicit  example  of 
independent  agents  with 
goals  that  conflicted  or  could 
interfere  with  the  DM’s  goals 

6.  Experience  to  complexity  factor  mapping:  PME  Revitalization 
Supervisor 
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Concept  Importance  for 
Scenario  (High,  Medium, 
Low)  and  Justification 

Connectivity: 

Things  in  the  environment 
influence  one  another  in 
complicated  and 

Examples  of  decisions  which  involved  interrelated  factors 
included: 

If  an  educational  program  is  being  changed,  the  changes 
have  to  be  made  with  consideration  of  all  elements  (army, 

Medium 

Factors  that  are  interrelated 
and  that  have  to  be 
considered  when  making 
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unpredictable  ways 

navy,  and  air  force)  to  ensure  that  the  changes  to  the 
system  or  course  are  consistent  with  needs  of  all 
elements  and  their  training  philosophies.  Because  of 
significant  differences  it  is  difficult  to  determine  what 
training  is  required,  how  it  should  be  done,  how  they  are 
going  to  get  buy-in  from  different  stakeholders,  etc.; 

If  something  is  added  to  a  DP,  then  something  else  may 
have  to  be  removed  and  this  could  cause  ripple  changes 
through  the  other  DP  levels  as  other  things  have  to  be 
shuffled  to  make  sure  that  things  are  learned  in  the  right 
order  and  prerequisites  are  maintained.  The  fact  that  the 
time  of  trainees  is  limited  also  creates  problems  with 
prioritizing  material  (usually  if  something  is  added 
something  else  has  to  be  dropped  to  fit  within  time 
constraints). 

decisions  (e.g.,  educational 
program  changes  involved 
at  least  6  interacting  factors 
that  the  DM  had  to  consider) 
Examples  of  2nd  and  3rd 
order  effects  (e.g.,  adding 
something  to  a  DP  could 
involve  considering  at  least 

3  other  DPs  and 
consideration  of  the  relative 
priority  of  many  constituent 
units  in  those  DPs) 

Dynamics: 

The  system  has  aspects 
that  unfold  over  time.  For 
example,  the  environment 
changes  over  time  even 
when  you  do  nothing;  the 
rate  at  which  things  change 
may  be  variable;  there  may 
be  delays  between  actions 
and  effects. 

It  is  important  to  note 
hierarchical  aspects  if 
present  (i.e.,  subsystems 
that  have  their  own 
dynamics  which  are  part  of 
the  DM  context). 

Situations  that  unfolded  over  time  that  profoundly  affected 
decision  making  included: 

Command  direction  slowed  the  achievement  of  goals  and 
more  scrutiny  was  paid  as  the  number  of  small 
information  requests  increased.  Whether  there  was 
general  risk  aversion  or  other  factors  were  at  play  is 
unknown; 

Meeting  requests  were  responded  to  less  favourably  over 
time,  which  delayed  the  process; 

The  DM’s  own  staff  had  difficulty  staying  motivated  and 
stress  increased  as  the  process  carried  on  over  time  with 
little  apparent  progress; 

Lack  of  continuity;  as  old  people  left  and  new  people 
entered  (including  the  DM)  the  process  it  meant  that  tasks 
were  performed  multiple  times,  people  had  to  be  gotten 
up  to  speed,  etc.; 

The  complexity  of  the  operating  environment  changed 
over  time.  The  Training  and  Education  (T&E)  system  is 
meant  to  keep  up  with  changes;  however,  the  T&E  system 
was  typically  fairly  slow  to  react  to  new  things,  and  so 
important  windows  of  opportunity  might  have  been 
missed; 

The  rate  at  which  things  changed  was  variable;  for 
example,  sometimes  submitting  information  to  the  Staff 
meant  that  the  DM  and  his  staff  were  bombarded  with 
requests  for  more  information,  and  sometimes  weeks 
would  go  by  with  no  feedback  from  the  Staff  at  all; 

Instructors  had  to  be  hired  to  run  programs;  this  had  to  be 
done  far  in  advance  (e.g.,  to  go  through  PWGSC 
process).  Due  to  delays  in  the  PME  process,  opportunities 
to  have  particular  instructors  at  particular  times  were  likely 
missed,  and  decisions  had  to  be  made  without  proper 

Medium 

Seven  specific  examples  of 
situations  that  unfolded  over 
time  that  profoundly  affected 
decision  making 

Four  subsystems  with 
somewhat  different 
dynamics  which  impacted 
decision  making 
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Concept  and  Definition 

Concept  Examples 

Concept  Importance  for 
Scenario  (High,  Medium, 
Low)  and  Justification 

support  from  a  finalized  PME  process. 

There  were  subsystems  which  had  their  own  dynamics  in 
this  context.  These  included: 

•  The  DM’s  team  (working  on  specific  tasks); 

•  The  School; 

•  The  Staff; 

•  The  Federal  government. 

•  The  School  and  the  Staff  seemed  to  have  their 

own  systems  and  timelines. 

•  The  overall  Federal  governmental  processes 
played  a  role  (e.g.,  the  Strategic  Review 
process). 

Multiple  conflicting  goals: 
Having  to  achieve  multiple 
objectives  which  may  not 
be  all  achievable  at  the 
same  time 

The  goals  of  the  DM  included: 

•  Increasing  the  number  of  staff  educated; 

•  Ensuring  the  education  program  met  the  needs 
of  the  CF; 

•  Meeting  budget  requirements 

•  Meeting  other  requirements  from  higher 
command  regarding  the  nature  of  what  and  who 
should  be  trained  (e.g.,  running  an  aboriginal 
program). 

It  was  impossible  to  meet  the  budget  requirements, 
increase  the  number  of  people  educated,  as  well  as 
running  all  of  the  other  programs  required.  It  was  unclear 
what  the  priorities  of  these  goals  should  be  and  they  could 
not  all  be  accomplished  at  the  same  time  (resources  were 
too  limited). 

Medium 

There  were  a  number  of 
important  goals  (4  distinct 
goals) 

Three  examples  of  goal 
conflict  provided 

Under-specified  goals: 

Goals  may  be  difficult  to 
achieve  because  they  are 
too  vague 

Examples  of  underspecified  goals  included: 

•  The  goals  of  running  the  joint  command  staff 
program  appeared  to  be  clear,  although  their 
relative  priority  were  not  always  clear. 

Low 

No  examples  of  vague  goals 
that  impacted  the  ability  of 
the  DM  to  make  decisions 

Goals  at  a  high  level 
seemed  to  be  fairly  clear. 

Independent  agents: 

There  are  independent 
entities  in  the  environment 
who  influence  it  (they  may 
have  different  goals  than 
the  decision  maker) 

Independent  agents  who  influenced  decision  making 
included: 

•  Each  environment  (army,  navy,  air  force):  each 
has  its  own  things  they  want  from  the  T&E 
system; 

•  The  School  personnel:  had  goal  of  obtaining 
money  for  out-of-scope  items  (e.g., 
infrastructure  improvements); 

•  Students  may  have  individual  constraints  (e.g., 

Medium 

Six  independent  agents 
who  impacted  decision 
making 

Two  examples  of 
independent  agents  with 
goals  that  conflicted  or  could 
interfere  with  the  DM’s  goals 
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senior  officers  may  be  extremely  time  limited 
with  regard  to  training);  many  have  to  balance 
education  with  a  full-time  position  and  family 
needs; 

•  The  Staff  (may  have  had  the  goal  of  delaying 
the  PME  project). 

The  goals  of  the  School  personnel  and  the  Staff  appeared 
to  directly  contradict  goals  of  the  DM.  Other  goals  had  to 
be  accommodated  during  planning. 

7.  Experience  to  complexity  factor  mapping:  CoE  Training 
Development  Supervisor 


Concept  and  Definition 

Concept  Examples 

Concept  Importance  for 
Scenario  (High,  Medium, 
Low)  and  Justification 

Connectivity: 

Things  in  the  environment 
influence  one  another  in 
complicated  and 
unpredictable  ways 

Examples  of  decisions  which  involved  interrelated  factors 
included: 

There  needs  to  be  overlap  and  continuity  in  training  (e.g., 
one  HQ  has  to  both  take  over  from  another  HQ  and  set 
the  conditions  for  the  next  HQ  to  come  in); 

Training  of  CF  personnel  also  involve  training  them  to 
deal  with  other  actors  including  members  of  DND,  DFAIT, 

Cl  DA,  Correctional  Services,  Policing  Associations,  NGOs 
-  the  DM  and  his  staff  have  to  put  in  a  lot  of  work  to  get 
these  groups  adequately  involved; 

The  request  to  include  logistics  personnel  in  the  training 
made  CANOSCOM  aware  that  more  logistics  preparation 
for  the  mission  was  required; 

There  was  a  request  to  inject  another  group  (high- 
readiness  HQ)  into  the  training  schedule.  This  had  follow- 
on  effects  such  as  having  to  plan  for  concurrent  training, 
having  to  re-distribute  the  workload,  and  new  people  had 
to  be  hired. 

Medium 

Multiple  factors  that  are 
interrelated  and  that  have  to 
be  considered  when  making 
decisions  (e.g.,  integrating 
other  actors  into  training 
involves  considering  six 
groups  that  the  DM  had  to 
consider) 

Examples  of  2nd  and  3rd 
order  effects  (e.g.,  the  need 
to  inject  another  group  into 
training  involved  at  least 
three  additional  decisions  or 
effects) 

Dynamics: 

The  system  has  aspects 
that  unfold  over  time.  For 
example,  the  environment 
changes  over  time  even 
when  you  do  nothing;  the 
rate  at  which  things  change 
may  be  variable;  there  may 
be  delays  between  actions 
and  effects. 

Situations  that  unfolded  over  time  that  profoundly  affected 
decision  making  included: 

Training  program  changed  over  time  as  different 
strategies  were  tried  as  the  opportunity  arose,  and  what 
worked  was  kept  or  improved  and  what  didn’t  work  was 
removed  or  changed; 

Early  attempts  were  made  to  integrate  organizations  such 
as  DFAIT,  CIDA,  and  CANOSCOM  into  the  training 
process  to  ensure  mutual  awareness  of  methods  and 
build  networks.  Over  time  it  became  apparent  that  these 

Medium 

Six  specific  examples  of 
situations  that  unfolded  over 
time  that  profoundly  affected 
decision  making 

Six  subsystems  with 
somewhat  different 
dynamics  which  impacted 
decision  making 
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It  is  important  to  note 
hierarchical  aspects  if 
present  (i.e.,  subsystems 
that  have  their  own 
dynamics  which  are  part  of 
the  DM  context). 

training  sessions  could  prove  very  beneficial  so  some 
organizations  became  more  involved  relatively  early  (e.g., 
DFAIT).  When  certain  training  goals  changed  to  become 
very  relevant  to  other  organizations  (e.g.,  CANOSCOM), 
then  they  became  involved  at  that  point; 

The  personnel  recruited  to  run  the  training  changed 
frequently  across  time.  A  new  training  focus  may  require  a 
change  in  SME  personnel.  This  requires  active 
recruitment  of  personnel  with  the  relevant  experience  to 
create  a  pool  of  people  who  can  be  drawn  upon  for 
training  needs.  There  is  always  a  need  to  find  recently 
retired  people  with  more  current  experience,  as  people 
who  have  been  retired  for  longer  may  have  lost  touch  with 
current  issues,  processes,  etc.  As  well,  for  each  individual 
occasion  people  have  to  be  scheduled  in  and  often  there 
are  fluid  schedule  constraints  based  on  other  people’s 
priorities  (holidays  etc.)  and  the  needs  of  other  projects 
(e.g.,  preparations  for  the  Olympics  required  a  large  staff); 
The  CF  staff  in  the  training  programs  change  cyclically 
and  frequently  over  time  (e.g.,  in  the  3.5  years  the  DM 
had  been  doing  this,  the  DM  had  4  bosses).  So,  certain 
things  need  to  be  done  over  and  over  again  (building 
relationships,  making  people  aware  of  how  things  have 
been  done,  what  worked,  what  didn’t,  etc.); 

There  was  a  request  to  inject  another  group  (high¬ 
readiness  HQ)  into  the  training  schedule.  This  had  follow- 
on  effects  such  as  having  to  plan  for  concurrent  training, 
having  to  re-distribute  the  workload,  and  new  people  had 
to  be  hired; 

The  training  programs  and  operational  plans  inform  one 
another  and  both  change  over  time.  For  example,  the 
writing  board  looks  for  weak  or  grey  areas  in  the  plan  to 
use  as  injects  for  training.  This  information  is  then  used  by 
the  planners  to  tighten  up  weak  spots  and  hopefully  avoid 
problems. 

There  were  subsystems  which  had  their  own  dynamics  in 
this  context.  These  included: 

•  Independent  contractors  (e.g.,  writing  board 
members); 

•  CDA  civilian  training  personnel  (the  DM  and  his 
staff); 

•  CF  training  personnel  at  CDA; 

•  DFAIT; 

•  Cl  DA; 

•  CANOSCOM. 

Groups  have  their  own  procedures  and  timelines  related 
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to  making  training  demands  or  supporting  training. 

Multiple  conflicting  goals: 

Having  to  achieve  multiple 
objectives  which  may  not 
be  all  achievable  at  the 
same  time 

The  goals  of  the  DM  included: 

•  Meeting  training  requirements  for  multiple 
groups 

•  Meeting  budget  constraints 

•  Building  a  network  of  resources 

•  Getting  buy-in  from  relevant  groups  and  getting 
their  involvement  as  required 

Resource  constraints  seem  to  be  the  main  source  of  goal 
conflict 

Medium 

There  were  four  distinct 
goals 

One  example  of  goal  conflict 

Under-specified  goals: 

Goals  may  be  difficult  to 
achieve  because  they  are 
too  vague 

Examples  of  underspecified  goals  included: 

•  Over  time  training  objectives  are  made  clear  but 
when  new  mission  priorities  occur  a 
development  process  for  creating  clear  training 
objectives  is  required. 

•  The  DM  usually  does  have  specific  training 
objectives  both  at  individual  and  HQ  level. 

Low 

No  examples  of  vague  goals 
that  impacted  the  ability  of 
the  DM  to  make  decisions 

Independent  agents: 

There  are  independent 
entities  in  the  environment 
who  influence  it  (they  may 
have  different  goals  than 
the  decision  maker) 

Independent  agents  who  influenced  decision  making 
included: 

•  Trainees 

•  ADM  materiel  (owner  of  all  equipment)  wants  to 
send  team  but  not  work  for  Cdr.,  wants  to  work 
for  Ottawa 

•  Independent  contractors  (e.g.,  writing  board 
members) 

•  CDA  civilian  training  personnel  (the  DM  and  his 
staff) 

•  OF  training  personnel  at  CDA 

•  DFAIT 

•  CIDA 

•  CANOSCOM 

Different  groups  have  different  hierarchies,  processes, 
goals,  priorities,  relationships,  and  see  their  role  in  this 
training  in  different  ways  from  one  another.  Different 
groups  play  different  roles  and  became  involved  at 
different  levels  and  at  different  times.  Ways  of  getting 
involvement  had  to  differ. 

Sometimes  the  DM  wants  involvement  of  individuals  or 
groups  which  are  not  supported  by  them  (e.g.,  ADM, 

DFAIT,  CIDA,  CANOSCOM) 

Medium 

Eight  independent  agents 
who  impacted  decision 
making 

Four  examples  of 
independent  agents  with 
goals  that  conflicted  with  the 
DM’s  goals 
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8.  Experience  to  complexity  factor  mapping:  HF  Engineer,  Capital 
Acquisition 


Concept  and  Definition 

Concept  Examples 

Concept  Importance  for 
Scenario  (High,  Medium, 
Low)  and  Justification 

Connectivity: 

Things  in  the  environment 
influence  one  another  in 
complicated  and 
unpredictable  ways 

Examples  of  decisions  which  involved  interrelated  factors 
included: 

Users  have  multiple  conflicting  needs.  Changing 
equipment  to  address  one  of  these  needs  will  affect  all  of 
the  other  needs  as  well  (either  positively  or  negatively); 

Feedback  from  industry  about  initial  requirements  caused 
a  lot  of  project  assumptions  to  be  questioned; 

The  DM  was  involved  in  other  similar  projects  and 
attempted  to  gain  some  needed  information  by  leveraging 
those  projects  to  get  information  needed  for  other  projects 
as  opportunities  arose; 

The  scope  of  the  project  changed  from  a  situation  in 
which  the  DM’s  team  would  work  with  the  winning  bidder 
to  develop  and  customize  a  system,  to  a  situation  where 
systems  the  bidders  presented  to  be  evaluated  were  what 
the  project  would  buy  for  the  CF  (i.e.,  no  CF 
customization).  This  created  a  huge  increase  in  risk, 
greatly  increased  the  required  specificity  for  the  statement 
of  requirements,  and  made  the  bid  evaluation  process 
much  more  important. 

Low 

There  are  interrelated 
factors  that  have  to  be 
considered  when  making 
decisions  (e.g.,  user  needs 
involved  at  least  3  factors 
that  the  DM  had  to 
consider). 

NOTE:  the  specific  needs 
mentioned  were  deleted  to 
maintain  confidentiality. 
Examples  of  2nd  and  3rd 
order  effects  (e.g.,  the  scope 
change  involved  at  least  3 
additional  decisions  or 
effects) 

Dynamics: 

The  system  has  aspects 
that  unfold  over  time.  For 
example,  the  environment 
changes  over  time  even 
when  you  do  nothing;  the 
rate  at  which  things  change 
may  be  variable;  there  may 
be  delays  between  actions 
and  effects. 

It  is  important  to  note 
hierarchical  aspects  if 
present  (i.e.,  subsystems 
that  have  their  own 
dynamics  which  are  part  of 
the  DM  context). 

Situations  that  unfolded  over  time  that  profoundly  affected 
decision  making  included: 

The  awareness  of  technology  that  was  available  changed 
over  the  course  of  the  project,  which  influenced  ideas 
about  what  was  possible  and  created  different 
opportunities; 

The  specification  of  requirements  was  an  iterative  process 
and  feedback  from  industry  caused  assumptions  about 
available  technology  to  be  re-evaluated  and  changed; 
Financial  resource  availability  for  HF  changed  over  time; 
initially  they  were  supposed  to  have  resources  for 

Research  and  Development  (R&D)  but  this  allocation  of 
resources  dropped  over  time  without  needed  R&D  being 
accomplished  (i.e.,  it  was  reallocated  for  other  non-HF 
purposes); 

The  project  proved  to  be  quick  to  spend  and  slow  to 
deliver;  the  strategic  review  and  other  forces  caused 
scope  changes,  resource  restrictions,  and  pressure  to 
deliver; 

The  scope  of  the  work  changed  repeatedly  over  time 
which  had  impacts  on  the  other  aspects  of  the  project. 

There  were  subsystems  which  had  their  own  dynamics  in 
this  context.  These  included: 

Medium 

Five  specific  examples  of 
situations  that  unfolded  over 
time  that  profoundly  affected 
decision  making 

Two  subsystems  with 
somewhat  different 
dynamics  which  impacted 
decision  making 
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•  The  other  decision  makers  in  the  project  were 
not  collocated  with  the  DM.  Information  flow  was 
different  between  these  two  groups. 

Multiple  conflicting  goals: 
Having  to  achieve  multiple 
objectives  which  may  not 
be  all  achievable  at  the 
same  time 

The  goals  of  the  DM  included: 

•  Creating  a  usable  and  valid  statement  of 
requirements; 

•  Meeting  resource  constraints; 

•  Conducting  necessary  R&D; 

•  Working  productively  in  the  team. 

These  goals  proved  difficult  to  achieve  at  the  same  time 
due  to  different  priorities  (e.g.,  difficult  to  create  statement 
of  requirements)  and  changing  requirements  (difficult  to 
create  requirements  while  meeting  resource  constraints). 

Medium 

There  were  four  important 
goals  that  conflicted  in 
different  ways  in  different 
situations 

Two  examples  of  goal 
conflict 

Primary  difficulty  due  to 
interpersonal  conflict  and 
changes  in  project  priorities 
and  requirements 

Under-specified  goals: 

Goals  may  be  difficult  to 
achieve  because  they  are 
too  vague 

Goals  were  not  necessarily  vague;  however,  project 
requirements  were  changed  frequently  leading  to 
confusion  and  wasted  effort. 

Low 

Goals  at  a  high  level  were 
fairly  clear,  the  main 
challenge  was  to  create 
concrete  measures  and 
subgoals 

Problems  were  created  by 
goal  changes  rather  than 
goals  being  unclear 

Independent  agents: 

There  are  independent 
entities  in  the  environment 
who  influence  it  (they  may 
have  different  goals  than 
the  decision  maker) 

Independent  agents  who  influenced  decision  making 
included: 

•  Other  team  members  who  didn’t  think  HF  issues 
were  important  and  didn't  want  them  included; 

•  Potential  bidders  (whether  they  would  be 
interested  in  bidding;  how  adequate  their 
equipment  would  be;  what  information  they 
needed); 

•  Users  (multiple  conflicting  needs); 

•  Members  of  the  DM’s  interpersonal  network 
(e.g.,  PMs  on  other  projects)  who  had  their  own 
goals  (e.g.,  validating  their  own  equipment). 

Low 

Four  independent  agents 
who  impacted  decision 
making 

One  example  of  an 
independent  agent  with 
goals  that  conflicted  or  could 
interfere  with  the  DM’s  goals 
Large  problems  due  to 
interpersonal  conflicts 

9.  Experience  to  complexity  factor  mapping:  PME  Revitalization 
Staff  Member 
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Concept  Importance  for 
Scenario  (High,  Medium, 
Low)  and  Justification 
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Concept  Importance  for 
Scenario  (High,  Medium, 
Low)  and  Justification 

Connectivity: 

Things  in  the  environment 
influence  one  another  in 
complicated  and 
unpredictable  ways 

Examples  of  decisions  which  involved  interrelated  factors 
included: 

If  a  training  program  is  being  changed,  the  DM  shouldn’t 
make  any  decisions  that  affect  policies  for  CF  programs 
without  consulting  all  elements  (army,  navy,  airforce); 

The  process  required  to  submit  proposals  related  to  PME 
Revitalization  changed  during  this  time  period.  As  the 
procedure  was  new,  there  was  a  general  lack  of  clarity 
about  expectations  and  difficulty  in  following  this  process; 

A  major  Strategic  Review  was  undertaken  that  affected 
the  whole  CF  during  this  time  period.  It  was  anticipated 
that  the  results  of  the  review  will  profoundly  affect  funding, 
including  funding  related  to  PME  Revitalization; 

There  was  difficulty  in  isolating  components  of  programs 
to  get  accurate  costing  information.  Proposals  for  funding 
generally  involve  many  interrelated  factors  such  as  actual 
cost  to  deliver  a  program,  infrastructure  and  overhead 
issues  (e.g.,  creation  of  a  virtual  library  is  required  for  a 
program  but  it  is  also  used  for  other  things  so  how  can 
you  determine  what  program  should  pay  what  proportion 
of  the  cost?). 

Low 

Multiple  factors  that  are 
interrelated  and  that  have  to 
be  considered  when  making 
decisions  (e.g.,  proposals  for 
funding  involved  at  least  3 
factors  that  the  DM  had  to 
consider 

Examples  of  2nd  and  3rd 
order  effects  (e.g.,  change  to 
proposals  for  the  PMB  had 
follow-on  effects  for  the 
process) 

Dynamics: 

The  system  has  aspects 
that  unfold  over  time.  For 
example,  the  environment 
changes  over  time  even 
when  you  do  nothing;  the 
rate  at  which  things  change 
may  be  variable;  there  may 
be  delays  between  actions 
and  effects. 

It  is  important  to  note 
hierarchical  aspects  if 
present  (i.e.,  subsystems 
that  have  their  own 
dynamics  which  are  part  of 
the  DM  context). 

Situations  that  unfolded  over  time  that  profoundly  affected 
decision  making  included: 

Improvements  and  changes  in  technology  (e.g.,  the 
possibility  of  creating  a  virtual  library)  put  different 
demands  on  and  created  different  opportunities  for  T&E; 

As  the  recession  occurred  and  got  more  serious  it  caused 
increasing  funding  concerns; 

As  the  impact  of  the  Canadian  operation  in  Afghanistan 
changed,  other  priorities  increased  (e.g.,  setting  up 
resource  centres  for  returning  soldiers,  supporting  military 
families,  supporting  the  health  and  welfare  of  returning 
soldiers)  so  projects  like  the  PME  revitalization  got 
pushed  to  a  lower  priority  and  delayed; 

Training  priorities  changed  over  time  as  senior 
commanders,  governments,  and  governmental  priorities 
changed  (e.g.,  more  or  less  demand  for  aboriginal 
programs); 

Things  would  go  quiet  for  a  few  months  and  then  there 
would  be  a  flurry  of  activity  (e.g.,  requests  for  more 
information)  which  would  require  a  response  and  then  it 
would  go  quiet  again;  this  appeared  to  the  DM  to  be  a 
cyclical  process; 

Meeting  requests  were  responded  to  less  favourably  over 
time,  which  created  delays  in  scheduling; 

Medium 

Seven  specific  examples  of 
situations  that  unfolded  over 
time  that  profoundly  affected 
decision  making 

Five  subsystems  with 
somewhat  different 
dynamics  which  impacted 
decision  making 
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Concept  and  Definition 

Concept  Examples 

Concept  Importance  for 
Scenario  (High,  Medium, 
Low)  and  Justification 

Lack  of  continuity,  with  new  people  entering  the  process, 
due  to  annual  postings,  meant  that  tasks  were  performed 
multiple  times  as  people  had  to  be  gotten  up  to  speed, 
etc. 

There  were  subsystems  which  had  their  own  dynamics  in 
this  context.  These  included: 

•  CF  educational  institutions; 

•  The  DM’s  institution; 

•  The  institution  that  was  to  help  prepare  the  PME 
proposal; 

•  Trainees; 

•  Instructors. 

There  were  different  subsystems  at  play  during  PME 
revitalization:  the  group  implementing  the  training  had 
different  timelines  and  rates  of  change  (e.g.,  represented 
in  rates  of  information  flow)  than  the  group  responsible  for 
guiding  the  training; 

The  trainees  and  instructors  work  at  a  longer  timeline  than 
the  other  CF  organizations  (e.g..,  they  need  more  notice 
than  was  actually  available  for  scheduling). 

Multiple  conflicting  goals: 
Having  to  achieve  multiple 
objectives  which  may  not 
be  all  achievable  at  the 
same  time 

The  goals  of  the  DM  included: 

•  Meeting  task  requirements; 

•  Meeting  budget  constraints; 

•  Effectively  using  limited  resources  (multiple 
programs  were  being  created  and  revised 
simultaneously  which  required  level  of  effort 
prioritization). 

If  goals  conflicted  it  was  usually  due  to  resource 
constraints  (i.e.,  not  enough  resources  to  meet  task 
requirements) 

Low 

There  were  3  important 
goals 

Goal  conflict  occurred, 
largely  due  to  resource 
constraints 

Under-specified  goals: 

Goals  may  be  difficult  to 
achieve  because  they  are 
too  vague 

Examples  of  underspecified  goals  included: 

•  The  DM  thought  the  goals  were  fairly  well 
defined  and  the  vision  from  superiors  was  clear. 
However,  often  the  DM  would  spend  a  lot  of 
time  on  a  project  and  then  it  would  get  “parked”; 
this  seems  to  indicate  a  mismatch  between 
priorities  and/or  lack  of  clarity  at  a  higher  level. 

Low 

No  explicit  cases  of  vague 
goals  were  provided. 

Goals  at  a  high  level  were 
fairly  clear  but  command 
intent  at  a  higher  level 
seemed  to  change  (or  be 
different  from  explicit 
statements  of  intent) 

Independent  agents: 

There  are  independent 
entities  in  the  environment 
who  influence  it  (they  may 

Independent  agents  who  influenced  decision  making 
included: 

•  The  staff  of  CF  educational  institutions; 

•  Other  Staff  at  the  DM’s  institution; 

Medium 

Six  independent  agents 
who  impacted  decision 
making 
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Concept  and  Definition 

Concept  Examples 

Concept  Importance  for 
Scenario  (High,  Medium, 
Low)  and  Justification 

have  different  goals  than 
the  decision  maker) 

•  Staff  at  the  institution  that  was  to  help  prepare 
the  PME  proposal; 

•  Trainees; 

•  Instructors; 

•  The  Federal  government. 

The  staff  of  CF  educational  institutions  had  their  own 
agenda  (e.g.,  fund  infrastructure  through  training 
projects); 

The  staff  at  the  institution  that  was  to  help  prepare  the 

PME  proposal  seemed  to  have  hidden  agendas  to  delay 
PME; 

The  trainees  have  individual  constraints  (e.g.,  students 
often  have  time  constraint  conflicts  between  their  job,  their 
family,  and  their  studies)  which  impact  their  ability  to 
complete  T&E,  which  in  turn  impacts  retention  rates; 

Instructors  have  their  own  agendas  and  availability  can 
impact  training  plans; 

The  Federal  government  added  additional  demands  (e.g., 
ministerial  inquiries  had  to  take  precedence  over  other 
work  when  they  occurred,  which  interfered  with  the  DM’s 
other  duties). 

Five  examples  of 
independent  agents  with 
goals  that  conflicted  or  could 
interfere  with  the  DM’s  goals 

10.  Experience  to  complexity  factor  mapping:  PSYOPS  Training 
Program  Developer 


Concept  and  Definition 

Concept  Examples 

Concept  Importance  for 
Scenario  (High,  Medium, 
Low)  and  Justification 

Connectivity: 

Things  in  the  environment 
influence  one  another  in 
complicated  and 
unpredictable  ways 

Examples  of  decisions  which  involved  interrelated  factors 
included: 

Multiple  factors  had  to  be  planned  which  impacted  one 
another  including  quarters,  rations,  course  content,  course 
schedules,  course  locations,  and  exercises; 

Because  of  the  short  timeline,  changes  in  one  resource 
(e.g.,  instructor  availability)  meant  that  things  had  to  be 
shifted  (e.g.,  the  order  of  instruction  changed); 

Two  group  members  had  significant  experience  in  close 
quarter  combat  training;  this  was  used  by  the  DM  and  two 
weeks  of  close  quarter  combat  training  was  offered 
internally.  This  offered  a  basics  review,  built  trust  within 
the  group,  increased  the  confidence  of  the  group,  and 
gave  the  trainees  an  additional  skill  set  to  increase  their 
value  to  the  other  personnel  they  were  stationed  with, 
which  in  turn  improved  their  chances  of  integrating  well. 

Medium 

Many  factors  that  are 
interrelated  and  that  have  to 
be  considered  when  making 
decisions  (e.g.,  planning 
includes  six  interacting 
variables  that  the  DM  had  to 
consider) 

Example  of  2nd  and  3rd  order 
effects  (e.g.,  close  quarter 
combat  training  resulted  in 
five  additional  effects) 
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Concept  and  Definition 

Concept  Examples 

Concept  Importance  for 
Scenario  (High,  Medium, 
Low)  and  Justification 

Dynamics: 

The  system  has  aspects 
that  unfold  over  time.  For 
example,  the  environment 
changes  over  time  even 
when  you  do  nothing;  the 
rate  at  which  things  change 
may  be  variable;  there  may 
be  delays  between  actions 
and  effects. 

It  is  important  to  note 
hierarchical  aspects  if 
present  (i.e.,  subsystems 
that  have  their  own 
dynamics  which  are  part  of 
the  DM  context). 

Situations  that  unfolded  over  time  that  profoundly  affected 
decision  making  included: 

Planning  for  the  training  programming  was  still  underway 
during  the  training  itself  (i.e.,  rolling-wave  planning).  For 
example  the  close-combat  training  was  arranged  after  the 
group  had  already  started  training  and  the  skills  of  the 
trainees  became  known.  This  allowed  the  DM  to  take 
advantage  of  opportunities  that  arose  but  was  not  optimal 
for  long-term  planning; 

Group  cohesion  was  an  important  process  for  facilitating 
training;  this  process  worked  well  in  this  situation  and  was 
impacted  by  factors  such  as  the  training  centre  requiring 
use  of  civilian  clothes,  trainees  boarding  together,  and 
team  members  conducting  close  quarter  combat  training. 

In  this  case,  the  group  gelled  together  well  (teambuilding 
was  considered  to  be  “outstanding”); 

Some  of  the  benefits  of  training  unfolded  over  time  rather 
than  being  immediately  apparent.  For  example,  some 
adversarial  intent  theory  was  provided  by  the  DRDC 
group.  The  usefulness  of  this  was  not  immediately 
apparent  to  the  trainees  but  the  relevance  and  importance 
was  understood  later  (e.g.,  when  they  had  to  defend  their 
ideas  in  the  field  they  could  apply  some  academic  rigor). 

There  were  subsystems  which  had  their  own  dynamics  in 
this  context.  These  included: 

•  There  were  several  subsystems  involved, 
including: 

•  The  CF  and  the  instructors. 

•  These  two  groups  had  their  own  processes  and 
timelines.  For  example,  the  CF  wanted  to 
arrange  the  training  on  a  short  timeline,  but 
many  instructors  were  booked  much  farther  in 
advance. 

Low 

Three  specific  examples  of 
situations  that  unfolded  over 
time  that  profoundly  affected 
decision  making 

Two  subsystems  with 
somewhat  different 
dynamics  which  impacted 
decision  making 

Multiple  conflicting  goals: 
Having  to  achieve  multiple 
objectives  which  may  not 
be  all  achievable  at  the 
same  time 

The  goals  of  the  DM  included: 

•  Meeting  training  requirements; 

•  Meeting  resource  requirements; 

•  Meeting  deadlines. 

•  Although  these  goals  could  work  against  one 
another  there  did  not  seem  to  be  a  large 
conflict;  the  main  issue  appeared  to  be  a  lack  of 
planning  lead  time. 

Low 

There  were  three  distinct 
goals 

Primary  difficulty  due  to  a 
lack  of  time  (i.e.,  resource 
conflicts) 

Under-specified  goals: 

Goals  may  be  difficult  to 
achieve  because  they  are 

Examples  of  underspecified  goals  included: 

•  The  goal  of  creating  the  best  PSYOPS  platoon 
possible  was  the  goal.  Ways  to  assess  this  and 

Low 

Goals  at  a  high  level  were 
fairly  clear,  the  main 

Page  120 


Humanly  .stems"  Incorporated 


Concept  and  Definition 

Concept  Examples 

Concept  Importance  for 
Scenario  (High,  Medium, 
Low)  and  Justification 

too  vague 

methods  to  achieve  this  had  to  be  developed. 

challenge  was  to  create 
concrete  measures  and 
subgoals 

Independent  agents: 

There  are  independent 
entities  in  the  environment 
who  influence  it  (they  may 
have  different  goals  than 
the  decision  maker) 

Independent  agents  who  influenced  decision  making 
included: 

•  Trainees  (they  had  different  backgrounds  and 
therefore  may  have  had  somewhat  different 
training  needs;  this  didn’t  really  conflict  with  the 
DM’s  goals  but  did  have  to  be  accommodated); 

•  Instructors  (their  schedules  dictated  when  they 
could  teach  their  material,  rather  than  optimal 
scheduling); 

•  Training  location  staff  (they  had  other  goals 
related  to  maintaining  a  civilian-centred 
institution  so  they  did  not  want  the  trainees  to 
wear  uniforms). 

Low 

Four  independent  agents 
who  impacted  decision 
making 

Two  examples  of 
independent  agents  with 
goals  that  conflicted  or  could 
interfere  with  the  DM’s  goals 
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Annex  D:  Bottom-up  Challenges  List 

Note  that  some  statements  include  challenges  or  training  suggestions  that  can  be  categorized  into 
more  than  one  general  theme.  These  have  been  entered  more  than  once  so  that  they  can  be  noted  as 
belonging  to  each  appropriate  theme.  Duplicated  challenges  and  training  suggestions  are 
highlighted  in  yellow  in  the  excel  file. 
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Initial  Order 


1 


2 

3 

4 


5 

6 


7 


8 

9 


10 


11 

Have  the  police  identify  some  of  their  upcoming  leaders  and  have  them  attend  CF  training  related  to 
higher  C2  so  when  these  events  take  place  you  have  familiarity  with  people,  with  language,  ongoing 

12  Mil  advisor:  international  even  exercises,  embedded  liaison 

13  Mil  advisor:  international  even  Difficult  to  manage  large  demand  for  information  flow 

14  Mil  advisor:  international  even  Location  of  the  event  was  not  optimal  for  security 

15  Mil  advisor:  international  even  The  personalities  of  some  of  the  people  involved  were  not  optimal  for  collaboration 

16  Mil  advisor:  international  even  The  event  was  a  highly  political  situation 

17  Mil  advisor:  international  even  Collaboration  made  difficult  because  of  different  jargon  used  by  the  different  organizations 

18  Mil  advisor:  international  even  Fencing  caused  problems  of  many  types,  including  the  perception  of  the  event,  deliveries,  etc. 

19  Mil  advisor:  international  even  There  were  substantial  misunderstandings  about  what  the  CF  were  willing  and  able  to  provide 

20  Mil  advisor:  international  even  Lack  of  previous  similar  events  to  use  for  planning  (e.g.,  no  lessons  learned) 

Status  of  security  force  attendance  was  highly  fluid  (e.g.,  what  was  happening  locally  could  have 

21  Mil  advisor:  international  even  meant  that  police  forces  would  or  would  not  send  forces  to  assist  with  security  for  the  event) 

22  Mil  advisor:  international  even  Budget  concerns 

23  Mil  advisor:  international  even  People  did  not  want  to  criticize  others  or  hear  negativity 

24  Mil  advisor:  international  even  People  were  highly  motivated  to  be  self-protective 

25  Mil  advisor:  international  even  No  known  ROEs  for  the  CF  if  they  had  to  act  in  this  situation  (e.g.,  to  defend  CF  assets  from  a  mob) 

26  Mil  advisor:  international  even  The  DM  had  no  authority  to  give  direction 

27  Mil  advisor:  international  even  Deliverables  kept  changing  (e.g.,  where  the  fence  would  be)  so  contracting  was  difficult 

28  Mil  advisor:  international  even  Negotiators  often  had  misunderstandings  about  the  process 

29  Mil  advisor:  international  even  Vital  legal  information  was  not  known  by  appropriate  authorities 

30  Mil  advisor:  international  even  Cases  where  individuals  had  personal  agendas  which  superseded  security  needs 

31  Mil  advisor:  international  even  Conflict  between  maintaining  pleasant  personal  relationships  and  getting  the  job  done 


After  the  prison  break  there  was  a  lot  of  distrust  of  the  prison  staff  and  a  reassessment  of  many 

Afgh  Liaison 

assumptions  about  the  state  of  security  in  Kandahar 

Challenge 

Planning 

Incorrect  assumptions 

Training 

Collaboration 

Challenge 

Information 

Challenge 

Location 

Challenge 

Collaboration 

Challenge 

Planning 

Challenge 

Communication 

Challenge 

Location 

Challenge 

Collaboration 

Challenge 

Planning 

Challenge 

Planning 

Challenge 

Resources 

Challenge 

Communication 

Challenge 

Collaboration 

Challenge 

Planning 

Challenge 

Authority 

Challenge 

Planning 

Challenge 

Collaboration 

Challenge 

Collaboration 

Challenge 

Collaboration 

Challenge 

Goal  conflict 

Extend  training  to  collaborators 
Overload 

Security  requirements 
Poor  collaborators 
Highly  political  situation 

Jargon 

Security  requirements 

Misunderstanding  roles  and  responsibilities 
Lack  of  previous  similar  events 


Lack  of  firm  plans 
Budget  concerns 
Lack  of  clarity  and  honesty 
Hidden  agendas 

Lack  of  previous  similar  events 

Responsibility  without  authority 

Uncertainty 

Lack  of  knowledge 

Lack  of  knowledge 

Hidden  agendas 

Achieve  multiple  conflicting  objectives 


Scenario 


Challenge/training  recommendation 


Challenge  or  Training  General  theme  Subtheme 


No  nexus  existed  for  communicating  CF  intelligence  to  other  organizations  (e.g.,  RCMP)  and  this  was  a 
serious  problem  as  it  was  illegal  to  communicate  information  from  CF  assets  that  had  to  do  with 
Mil  advisor:  international  even  conducting  surveillance  of  Canadians  on  Canadian  soil  Challenge 

No  nexus  existed  for  communicating  CF  intelligence  to  other  organizations  (e.g.,  RCMP)  and  this  was  a 
serious  problem  as  it  was  illegal  to  communicate  information  from  CF  assets  that  had  to  do  with 
Mil  advisor:  international  even  conducting  surveillance  of  Canadians  on  Canadian  soil  Challenge 

Mil  advisor:  international  even  Specific  terms  of  reference  might  have  made  his  job  easier  (although  flexibility  was  needed)  Challenge 

Mil  advisor:  international  even  Specific  terms  of  reference  might  have  made  his  job  easier  (although  flexibility  was  needed)  Challenge 


Planning 

Communication 

Collaboration 

Direction 


Insufficient  planning 

Procedure 
Flexibility  required 
Vague 


If  he  had  been  upranked  one  that  would  have  helped  (replaced  someone  who  was  a  rank  above  and 
Mil  advisor:  international  even  the  perception  was  that  his  job  wasn't  important)  Challenge 

It  would  have  helped  if  he  had  been  brought  in  sooner  (could  have  built  up  relationships  more  in 
Mil  advisor:  international  even  advance)  Challenge 


Authority 

Collaboration 


Not  enough 
Insufficient  lead  time 


Make  sure  you  have  access  to  SMEs  and  that  you  listen  to  them  -  experienced  SMEs  who  can  walk  you 
Mil  advisor:  international  even  though  case  studies  and  case  analyses  of  previous  situations  -  use  the  knowledge  of  previous  events  Training 
Need  to  objectively  analyse  all  of  the  steps  that  were  taken  and  the  OPP  that  was  used  -  compare 
plans  to  the  actual  operation,  identify  what  worked,  examine  whether  the  right  assumptions  were 
Mil  advisor:  international  even  made  Training 

Make  sure  you  use  the  right  tool  at  the  right  time  -  e.g.,  the  IPP  is  often  more  appropriate  that  the 
Mil  advisor:  international  even  OPP  Training 


Experience 

Evaluation 

Planning 


Objective  assessment  required 
Right  tool  at  right  time 


Need  the  right  person  in  the  right  job  -  need  the  right  experience  and  the  right  personality  -  e.g., 

CIMIC  is  a  reserve  capacity  for  the  CF  because  those  people  know  how  to  walk  both  sides  of  the  fence  - 
Mil  advisor:  international  even  need  a  certain  amount  of  maturity  and  respect  for  relationships  Training 

Need  the  right  person  in  the  right  job  -  need  the  right  experience  and  the  right  personality  -  e.g., 

CIMIC  is  a  reserve  capacity  for  the  CF  because  those  people  know  how  to  walk  both  sides  of  the  fence  - 
Mil  advisor:  international  even  need  a  certain  amount  of  maturity  and  respect  for  relationships  Training 


Experience 


Personality 


Right  person  in  right  job 


Right  person  in  right  job 


Initial  Order 

Scenario 

Challenge/training  recommendation 

Challenge  or  Training 

General  theme 

Subtheme 

33 

Afgh  Liaison 

After  the  prison  break  there  was  a  lot  of  distrust  of  the  prison  staff  and  a  reassessment  of  many 
assumptions  about  the  state  of  security  in  Kandahar 

Challenge 

Collaboration 

Distrust 

34 

Afgh  Liaison 

Many  challenges  related  to  cultural  issues,  including  different  priorities  (interpersonal  relationships 
highest  in  Afghanistan,  age  more  indicative  of  authority),  differences  in  the  legal  system,  widespread 
corruption,  different  understandings  of  jobs  and  job  responsibilities,  different  beliefs  about  the 
amount  of  control  one  can  exert  over  a  situation,  different  social  standards,  differences  in  literacy 
rates  (few  written  records  in  Afghanistan),  and  a  need  for  Afghans  to  save  face 

Challenge 

Culture 

Different  priorities 

35 

Afgh  Liaison 

Many  challenges  related  to  cultural  issues,  including  different  priorities  (interpersonal  relationships 
highest  in  Afghanistan,  age  more  indicative  of  authority),  differences  in  the  legal  system,  widespread 
corruption,  different  understandings  of  jobs  and  job  responsibilities,  different  beliefs  about  the 
amount  of  control  one  can  exert  over  a  situation,  different  social  standards,  differences  in  literacy 
rates  (few  written  records  in  Afghanistan),  and  a  need  for  Afghans  to  save  face 

Challenge 

Culture 

Legal  system 

36 

Afgh  Liaison 

Many  challenges  related  to  cultural  issues,  including  different  priorities  (interpersonal  relationships 
highest  in  Afghanistan,  age  more  indicative  of  authority),  differences  in  the  legal  system,  widespread 
corruption,  different  understandings  of  jobs  and  job  responsibilities,  different  beliefs  about  the 
amount  of  control  one  can  exert  over  a  situation,  different  social  standards,  differences  in  literacy 
rates  (few  written  records  in  Afghanistan),  and  a  need  for  Afghans  to  save  face 

Challenge 

Culture 

Corruption 

37 

Afgh  Liaison 

Many  challenges  related  to  cultural  issues,  including  different  priorities  (interpersonal  relationships 
highest  in  Afghanistan,  age  more  indicative  of  authority),  differences  in  the  legal  system,  widespread 
corruption,  different  understandings  of  jobs  and  job  responsibilities,  different  beliefs  about  the 
amount  of  control  one  can  exert  over  a  situation,  different  social  standards,  differences  in  literacy 
rates  (few  written  records  in  Afghanistan),  and  a  need  for  Afghans  to  save  face 

Challenge 

Culture 

Different  understanding  of  jobs  &  responsibilities 

38 

Afgh  Liaison 

Many  challenges  related  to  cultural  issues,  including  different  priorities  (interpersonal  relationships 
highest  in  Afghanistan,  age  more  indicative  of  authority),  differences  in  the  legal  system,  widespread 
corruption,  different  understandings  of  jobs  and  job  responsibilities,  different  beliefs  about  the 
amount  of  control  one  can  exert  over  a  situation,  different  social  standards,  differences  in  literacy 
rates  (few  written  records  in  Afghanistan),  and  a  need  for  Afghans  to  save  face 

Challenge 

Culture 

Different  belief  in  individual  control 

39 

Afgh  Liaison 

Many  challenges  related  to  cultural  issues,  including  different  priorities  (interpersonal  relationships 
highest  in  Afghanistan,  age  more  indicative  of  authority),  differences  in  the  legal  system,  widespread 
corruption,  different  understandings  of  jobs  and  job  responsibilities,  different  beliefs  about  the 
amount  of  control  one  can  exert  over  a  situation,  different  social  standards,  differences  in  literacy 
rates  (few  written  records  in  Afghanistan),  and  a  need  for  Afghans  to  save  face 

Challenge 

Culture 

Social  standards 

40 

Afgh  Liaison 

Many  challenges  related  to  cultural  issues,  including  different  priorities  (interpersonal  relationships 
highest  in  Afghanistan,  age  more  indicative  of  authority),  differences  in  the  legal  system,  widespread 
corruption,  different  understandings  of  jobs  and  job  responsibilities,  different  beliefs  about  the 
amount  of  control  one  can  exert  over  a  situation,  different  social  standards,  differences  in  literacy 
rates  (few  written  records  in  Afghanistan),  and  a  need  for  Afghans  to  save  face 

Challenge 

Culture 

Literacy 

41 

42 

43 

Afgh  Liaison 

Afgh  Liaison 

Afgh  Liaison 

Many  challenges  related  to  cultural  issues,  including  different  priorities  (interpersonal  relationships 
highest  in  Afghanistan,  age  more  indicative  of  authority),  differences  in  the  legal  system,  widespread 
corruption,  different  understandings  of  jobs  and  job  responsibilities,  different  beliefs  about  the 
amount  of  control  one  can  exert  over  a  situation,  different  social  standards,  differences  in  literacy 
rates  (few  written  records  in  Afghanistan),  and  a  need  for  Afghans  to  save  face 

Communication  infrastructure  is  poor  in  Afghanistan 

Communication  infrastructure  is  poor  in  Afghanistan 

Challenge 

Challenge 

Challenge 

Culture 

Communication 

Resources 

Saving  face 

Infrastructure 

Communication  infrastructure 

44  Afgh  Liaison 

ANA  and  ANP  have  an  adversarial  relationship  and  do  not  work  well  together 

Challenge 

Collaboration 

Adversarial  relationships 

45  Afgh  Liaison 

Quick  turnovers  in  Afghan  personnel  make  it  difficult  to  get  to  know  people  and  form  relationships 

Challenge 

Collaboration 

Turnover 

46 

Afgh  Liaison 

Delicate  balance  between  giving  accurate  and  helpful  feedback  and  making  people  demotivated 

Challenge 

Collaboration 

Motivation 

47 

Afgh  Liaison 

Delicate  balance  between  giving  accurate  and  helpful  feedback  and  making  people  demotivated 

Challenge 

Goal  conflict 

Achieve  multiple  conflicting  objectives 

48  Afgh  Liaison 

Lack  of  reliable  power  in  Afghanistan 

Challenge 

Resources 

Power  infrastructure 

Initial  Order 

Scenario 

Challenge/training  recommendation 

Challenge  or  Training 

General  theme 

Subtheme 

49  Afgh  Liaison 

No  common  COP  among  Afghan  security  agencies 

Challenge 

Collaboration 

No  COP 

50  Afgh  Liaison 

Found  collaborators  highly  resistant  to  change 

Challenge 

Collaboration 

Change  resistance 

51 

Afgh  Liaison 

Difficult  to  get  correct  information  and  difficult  to  evaluate  information 

Challenge 

Information 

Difficult  to  get  correct  information 

52 

Afgh  Liaison 

Difficult  to  get  correct  information  and  difficult  to  evaluate  information 

Challenge 

Information 

Difficult  to  evaluate 

Official  positions  often  contradict  what  actually  happens  (e.g.,  told  you  have  authority  to  do 

53 

Afgh  Liaison 

something  but  when  try  to  do  it  you  are  denied) 

Challenge 

Collaboration 

Hidden  agendas 

54 

Afgh  Liaison 

Afghans  think  at  tactical  level  only 

Challenge 

Culture 

Lack  of  knowledge 

55 

Afgh  Liaison 

Different  ideas  of  how  to  train  and  what  training  is  (e.g.,  no  experience  with  exercises) 

Challenge 

Culture 

Lack  of  knowledge 

56 

Afgh  Liaison 

Afghans  didn't  understand  that  you  should  use  data  to  make  conclusions 

Challenge 

Culture 

Lack  of  knowledge 

57 

Afgh  Liaison 

Afghan  intel  was  not  good  at  instructing  patrols  what  they  should  look  for 

Challenge 

Communication 

Vague 

58 

Afgh  Liaison 

Often  different  people  would  come  to  meetings  -  low  continuity  in  personnel 

Challenge 

Collaboration 

Turnover 

59 

Afgh  Liaison 

Sometimes  people  wouldn't  want  to  share  information  because  it  would  get  them  into  trouble 

Challenge 

Collaboration 

Hidden  agendas 

60 

Afgh  Liaison 

Lots  of  demands  for  resources  from  people  who  couldn't  really  help  the  DM’s  mission 

Challenge 

Resources 

61 

Afgh  Liaison 

Had  to  convince  Afghans  that  a  security  network  was  even  necessary 

Challenge 

Role  justification 

62 

Afgh  Liaison 

People  needed  to  mentor  the  Afghans  about  operational  rather  than  just  tactical  level 

The  enemy  has  at  least  some  personnel  better  at  strategic  and  operational  level  thinking  than  the 

Training 

Culture 

Lack  of  knowledge 

63 

Afgh  Liaison 

Afghan  security  forces  allied  with  the  CF 

Challenge 

Planning 

64 

Afgh  Liaison 

The  DM  had  to  deal  with  multiple  Afghan  languages 

Challenge 

Culture 

Language 

65 

Afgh  Liaison 

Had  the  need  to  appear  to  maintain  enthusiasm  to  keep  others  motivated  and  involved 

Challenge 

Collaboration 

Motivation 

66 

Capital  Acqusition 

Different  perspectives  between  team  members 

Challenge 

Collaboration 

Different  perspectives 

67 

Capital  Acqusition 

Lack  of  appreciation  for  DM’s  contribution  (value  of  HF) 

Challenge 

Role  justification 

68 

Capital  Acqusition 

Increased  workload  due  to  differences  of  opinion  about  value  of  HF 

Challenge 

Collaboration 

Workload 

69 

Capital  Acqusition 

Other  personnel  resistant  to  change  of  opinion 

Challenge 

Collaboration 

Change  resistance 

70 

Capital  Acqusition 

Insufficient  data  available  to  make  requirements  recommendations 

Challenge 

Information 

Insufficient  data 

71 

Capital  Acqusition 

DM  not  collocated  with  other  decision  makers  which  impacted  communication 

Challenge 

Collaboration 

Collocation 

72 

Capital  Acqusition 

Difficult  to  share  some  information  with  bidders  as  it  is  confidential 

Challenge 

Communication 

Procedure 

73 

Capital  Acqusition 

Difficult  to  share  equipment  with  bidders  as  it  is  being  used 

Challenge 

Resources 

Lack  of  equipment 

74 

Capital  Acqusition 

Difficult  to  create  a  testing  baseline 

Users  have  multiple  conflicting  needs;  making  changes  will  almost  inevitably  affect  multiple  needs 

Challenge 

Evaluation 

No  baseline 

75 

Capital  Acqusition 

both  positively  and  negatively 

Resources  were  not  used  optimally  and  decreased  resources  available  for  needs  identified  by  the  DM 

Challenge 

Goal  conflict 

Consider  multiple  factors 

76 

Capital  Acqusition 

(e.g.,  research  to  determine  HF  requirements) 

Challenge 

Resources 

Budget 

77 

Capital  Acqusition 

The  credibility  of  the  DM  was  questioned  based  on  decisions  not  made  by  them 

Challenge 

Collaboration 

Credibility 

78 

Capital  Acqusition 

There  was  resistance  to  including  testing  for  all  important  interacting  factors 

There  was  no  strategic  plan  put  in  place  to  control  the  number  of  bidders  and  so  there  turned  out  to 

Challenge 

Information 

Insufficient  data 

79 

Capital  Acqusition 

be  more  bidders  than  could  easily  be  handled  in  bid  evaluation 

Scope  changes  increased  risk  and  required  specificity  for  SOR,  increased  pressure  on  bid  evaluation 

Challenge 

Planning 

Insufficient  planning 

80 

Capital  Acqusition 

process 

Challenge 

Planning 

Scope  changes 

81 

Capital  Acqusition 

Awareness  of  available  technology  changed  over  the  life  of  the  project 

Challenge 

Information 

Changing  information 

82 

Capital  Acqusition 

Financial  resource  availability  for  HF  changed  over  time  without  needed  R&D  being  accomplished 

Challenge 

Resources 

Budget 

83 

Capital  Acqusition 

Pressure  to  deliver  rose  over  time 

Challenge 

Resources 

Timeline 

84 

Capital  Acqusition 

Scope  of  work  changed  repeatedly  over  time  which  had  impacts  on  other  project  aspects 

Should  have  used  high-level  specification  rather  than  very  detailed  ones  and  maintained  maturation 

Challenge 

Planning 

Scope  changes 

85 

Capital  Acqusition 

phase 

Training 

Planning 

Planning  incorrect 

86 

Capital  Acqusition 

Need  to  improve  when  resources  brought  on  board  relative  to  when  they  are  actually  needed 

Training 

Resources 

Planning 

87 

Capital  Acqusition 

Reduce  collocation  issues  (management  team  should  be  collocated  if  possible) 

Training 

Collaboration 

Collocation 

88 

Capital  Acqusition 

Need  to  take  advantage  of  multiple  related  projects  and  achieve  multiple  goals  from  multiple  projects  Training 

Resources 

Achieve  multiple  objectives 

Sometimes  can  use  social  network  to  facilitate  interactions  with  team  members  and  other 

89 

Capital  Acqusition 

collaborators 

Training 

Collaboration 

Networking 

90 

Capital  Acqusition 

Give  incoming  staff  history  of  team  members  to  prepare  them  for  likely  challenges 

Training 

Collaboration 

Sharing  information 

91 

Capital  Acqusition 

Perhaps  an  outside  HF  consultant  would  have  had  a  bigger  impact  than  someone  in  the  same  team 

Training 

Role  justification 

92 

Capital  Acqusition 

Heavily  prepared  presentations  for  meetings  with  basic  justifications  (sometimes  over  and  over) 

Training 

Role  justification 

93 

Capital  Acqusition 

Use  empirical  evidence  if  available 

Training 

Evaluation 

Use  empirical  evaluation 

94 

Capital  Acqusition 

Know  who  the  players  are  on  the  team 

Training 

Collaboration 

Know  your  team 

Order 

Scenario 

Challenge/training  recommendation 

Challenge  or  Training 

General  theme 

Subtheme 

95 

Capital  Acqusition 

Teach  basic  arguments  for  justifying  HF 

Training 

Role  justification 

96 

Capital  Acqusition 

Make  sure  you  know  the  current  state  of  knowledge  -  what  other  team  members  know 

Training 

Collaboration 

Know  your  team 

97 

Capital  Acqusition 

Know  how  and  what  to  communicate 

Have  a  network  of  contacts  (e.g.,  from  conferences,  trade  shows,  literature)  -  look  beyond  the  obvious 

Training 

Communication 

98 

Capital  Acqusition 

for  information 

Don't  use  your  own  experience  and  intuition  alone  when  evaluating  designs  -  often  people  don’t  like 

Training 

Collaboration 

Networking 

99 

Capital  Acqusition 

designs  that  they  think  they  will 

Changes  to  programs  should  be  done  in  consultation  with  a  large  number  of  stakeholders  (e.g.,  all 

Training 

Evaluation 

Use  empirical  evaluation 

100 

PME  Staff 

elements) 

Challenge 

Goal  conflict 

Consider  multiple  factors 

The  process  required  to  submit  proposals  related  to  PME  Revitalization  changed;  there  was  a  general 

101 

PME  Staff 

lack  of  clarity  about  expectations  and  difficulty  related  to  the  new  process 

The  process  required  to  submit  proposals  related  to  PME  Revitalization  changed;  there  was  a  general 

Challenge 

Planning 

Procedure  changes 

102 

PME  Staff 

lack  of  clarity  about  expectations  and  difficulty  related  to  the  new  process 

Challenge 

Direction 

Vague 

103 

PME  Staff 

Major  strategic  review  occurred  which  is  anticipated  to  profoundly  affect  funding 

Challenge 

Resources 

Funding  review 

104 

PME  Staff 

Disagreements  between  stakeholders  regarding  scope  of  funding 

Challenge 

Collaboration 

Disagreements 

105 

PME  Staff 

Disagreements  between  stakeholders  regarding  scope  of  funding 

Challenge 

Resources 

Inconsistent  expectations 

106 

PME  Staff 

Accounting  errors  in  database 

Proposals  for  funding  involve  many  interrelated  factors  -  difficult  to  isolate  some  costs  to  one  program 

Challenge 

Information 

Errors 

107 

PME  Staff 

alone  (e.g.,  virtual  libraries) 

Challenge 

Information 

Interrelated  factors 

108 

PME  Staff 

Lack  of  timely  feedback  after  information  submitted 

Challenge 

Evaluation 

Lack  of  feedback 

109 

PME  Staff 

Briefings  must  be  thorough  yet  concise  because  those  being  briefed  have  limited  time 

Challenge 

Resources 

Time  limitations 

110 

PME  Staff 

Additional  requirements  often  do  not  come  with  additional  funds 

Challenge 

Resources 

Budget 

111 

PME  Staff 

Changes  in  technology  puts  different  demands  on  and  creates  different  opportunities  for  T&E 

Challenge 

Resources 

Changes  in  available  resources  and  consequences 

112 

PME  Staff 

Recession  created  additional  funding  pressure 

The  prioritization  of  programs  like  PME  is  influenced  by  other  strategic  priorities  out  of  the  control  of 

Challenge 

Resources 

Budget 

113 

PME  Staff 

the  DM  (due  to  changing  operations,  changing  governments,  etc.) 

Unpredictable  requests  for  information  that  have  to  be  filled  quickly;  these  alternate  with  periods  of 

Challenge 

Planning 

114 

PME  Staff 

silence. 

Challenge 

Information 

Workload 

115 

PME  Staff 

Lack  of  continuity  in  personnel 

Challenge 

Collaboration 

Turnover 

116 

PME  Staff 

Planning  for  training  programs  had  to  be  done  in  advance  of  funding  certainty 

Should  approach  data  collection  related  to  funding  needs  from  first  principles  rather  than  relying  on 

Challenge 

Planning 

Lack  of  required  information 

117 

PME  Staff 

previously  compiled  data 

Training 

Information 

First  principles 

118 

PME  Staff 

When  new  programs  begin  collect  new  data  and  start  from  scratch 

Training 

information 

New  data 

119 

PME  Staff 

Need  face  to  face  meetings 

Training 

Collaboration 

Collocation 

120 

PME  Staff 

Need  to  work  on  getting  everyone  speaking  the  same  language 

Training 

Communication 

Jargon 

The  DM  tried  to  meet  demands  for  requirements  but  they  kept  coming  back  and  asking  for  more  and 

121 

PME  Staff 

for  information  to  be  presented  in  different  ways 

The  DM  tried  to  meet  demands  for  requirements  but  they  kept  coming  back  and  asking  for  more  and 

Challenge 

Collaboration 

Inconsistent  expectations 

122 

PME  Staff 

for  information  to  be  presented  in  different  ways 

Challenge 

Collaboration 

Hidden  agendas 

123 

PME  Staff 

Need  to  make  sure  all  important  people  at  meetings 

Training 

Collaboration 

Get  proper  people  involved 

124 

PME  Supervisor 

Instructed  to  begin  training  before  funding  terms  available 

Changes  to  programs  need  to  be  made  in  consideration  of  many  stakeholders  with  different 

Challenge 

Resources 

Uncertainty 

125 

PME  Supervisor 

philosophies 

Changes  to  one  aspect  of  a  training  program  has  to  take  into  account  past  and  future  T&E  (e.g.,  the  DP 

Challenge 

Goal  conflict 

Consider  multiple  factors 

126 

PME  Supervisor 

education  packages  are  interdependent) 

Challenge 

Planning 

Interdependence 

127 

PME  Supervisor 

Time  of  trainees  is  limited,  so  training  package  size  is  limited 

Challenge 

Resources 

Time 

128 

PME  Supervisor 

Disagreements  between  stakeholders  regarding  scope  of  funding 

Challenge 

Collaboration 

Disagreements 

129 

PME  Supervisor 

Previous  databases  of  funding  requirements  were  padded 

Challenge 

Collaboration 

Dishonesty 

130 

PME  Supervisor 

Stakeholders  were  overstepping  their  range  of  authority 

Challenge 

Authority 

Overstepping 

131 

PME  Supervisor 

Lack  of  clear  command  intent  within  other  organizations 

Challenge 

Direction 

Vague 

132 

PME  Supervisor 

Periods  of  silence  of  significant  duration 

Challenge 

Evaluation 

Lack  of  feedback 

133 

PME  Supervisor 

Requirements  for  new  training  programs  are  given  without  additional  resources 

Requests  for  clarification  from  superiors  about  prioritization  of  programs  did  not  result  in  clear 

Challenge 

Resources 

Budget 

134 

PME  Supervisor 

direction 

Challenge 

Direction 

Vague 

135 

PME  Supervisor 

Strategic  Review  is  likely  to  profoundly  affect  funding 

Challenge 

Resources 

Funding  review 

136 

PME  Supervisor 

Pushing  something  up  the  authority  hierarchy  for  resolution  means  that  the  DM  risks  delays  etc. 
Requirement  to  balance  short-term  with  long-term  goals  (e.g.,  using  authority  can  get  your  way  in  the 

Challenge 

Authority 

Loss  of  control 

137 

PME  Supervisor 

short  term  but  poisons  relationships) 

Challenge 

Goal  conflict 

Consider  long-term  effects 

Initial  Order  Scenario 

Challenge/training  recommendation 

Challenge  or  Training 

General  theme 

Subtheme 

138  PME  Supervisor 

No  in-year  funding  available,  so  resources  even  more  restricted 

Challenge 

Resources 

Budget 

139  PME  Supervisor 

Impression  that  command  intent  to  subordinates  was  actually  to  delay  the  process 

Challenge 

Collaboration 

Hidden  agendas 

140  PME  Supervisor 

Meeting  requests  responded  to  less  favourably  over  time 

Challenge 

Collaboration 

Hidden  agendas 

141  PME  Supervisor 

Lack  of  continuity  in  personnel  -  people  have  to  be  gotten  up  to  speed,  etc. 

T&E  system  meant  to  keep  up  with  rapid  changes  in  operational  environment;  however,  T&E  process 

Challenge 

Collaboration 

Turnover 

142  PME  Supervisor 

is  typically  actually  slow  to  adapt  and  change 

Unpredictable  response  after  information  exchange  (e.g.,  silence  or  bombarded  with  more 

Challenge 

Planning 

Lead  time  required  is  too  long 

143  PME  Supervisor 

information  requests) 

Challenge 

Information 

Workload 

144  PME  Supervisor 

Planning  for  training  programs  had  to  be  done  in  advance  of  funding  certainty 

Challenge 

Planning 

Lack  of  required  information 

145 

146 

PME  Supervisor 

PME  Supervisor 

Given  instructions  from  higher  command  to  do  things  without  any  funds  available 

Given  instructions  from  higher  command  to  do  things  without  any  funds  available 

Challenge 

Challenge 

Direction 

Resources 

Impossible  task 

Budget 

147 

PME  Supervisor 

Has  to  manage  stress  on  staff  from  them  having  to  work  on  something  and  then  having  it  shelved 

Challenge 

Collaboration 

Morale  and  stress 

148 

PME  Supervisor 

Subordinate  commanders  have  to  take  on  more  risk  than  they  should  due  to  lack  of  direction  from 
command 

Challenge 

Direction 

Vague 

149 

PME  Supervisor 

Most  decisions  the  DM  made  were  intuitive  based  on  experience  and  how  you  work  with  people  and 
can  try  to  move  things  forward 

Training 

Experience 

Use  intuition 

150 

151 

PME  Supervisor 

PME  Supervisor 

Most  problems  seemed  to  be  due  to  communication  issues 

Should  have  trigger  points  in  place  for  when  follow-up  or  other  actions  required 

Challenge 

Training 

Communication 

Planning 

Use  trigger  points  in  plans 

152 

PME  Supervisor 

Need  clear  intent  recorded  as  well  as  records  of  decisions  (e.g.,  who  told  what  to  do  what  on  what 
date) 

Training 

Direction 

Need  clear  direction 

153 

PME  Supervisor 

Need  clarity  of  words,  intent,  effect,  etc. 

Training 

Direction 

Need  clear  direction 

154 

PME  Supervisor 

Stakeholders  had  different  interpretations  of  one-line  objective/intent  which  should  have  been 
clarified 

Training 

Direction 

Vague 

155 

PME  Supervisor 

Stakeholders  had  different  interpretations  of  one-line  objective/intent  which  should  have  been 
clarified 

Training 

Collaboration 

Different  understanding 

156 

PME  Supervisor 

Terminology  had  to  be  agreed  upon 

Training 

Communication 

Jargon 

157 

PME  Supervisor 

Comes  down  to  making  sure  all  communication  is  clear 

Training 

Communication 

Clarity 

158 

PME  Supervisor 

Can  be  a  problem  that  superiors  don't  give  very  concrete  and  clear  intent  -  incumbent  on 
subordinates  to  go  back  and  clarify 

Training 

Direction 

Vague 

159 

PME  Supervisor 

Can  be  a  problem  that  superiors  don't  give  very  concrete  and  clear  intent  -  incumbent  on 
subordinates  to  go  back  and  clarify 

Training 

Direction 

Need  to  get  clarification 

160 

161 

PME  Supervisor 

PME  Supervisor 

Realize  that  if  you  are  seen  as  obstructionist  people  will  learn  to  work  around  you 

Be  sure  to  engage  higher  levels  at  proper  points  to  ensure  best  effect 

Training 

Training 

Collaboration 

Authority 

Engage  higher  authority  at  proper  time 

162 

PME  Supervisor 

Make  sure  to  integrate  with  your  staff  to  make  sure  they  stay  on  top  of  things  you  are  interested  in 

Training 

Collaboration 

Teamwork 

163 

PME  Supervisor 

Keep  staff  engaged  by  remaining  engaged  yourself 

Training 

Collaboration 

Leadership 

164 

PME  Supervisor 

Most  effective  problem  solving  was  when  everyone  was  brought  together 

Training 

Collaboration 

Collocation 

165 

Strategic  Advisory  Team 

Difficult  to  get  timely  and  accurate  information  -  no  "ground  truth" 

Challenge 

Information 

Inadequate 

166 

Strategic  Advisory  Team 

Difficult  to  get  timely  and  accurate  information  -  no  "ground  truth" 

Challenge 

Information 

Inaccurate 

167 

Strategic  Advisory  Team 

News  can  travel  quickly  in  the  Afghan  population,  adding  to  the  risk  of  riots 

Challenge 

Communication 

Speed  of  information  travel  among  civilians 

168 

Strategic  Advisory  Team 

Communications  back  to  Canada  and  to  TFA  were  not  reliable 

Challenge 

Communication 

Unreliable  infrastructure 

169 

Strategic  Advisory  Team 

Heavily  influenced  by  Afghan  actions  against  U.S.  and  other  nearby  embassies  etc. 

Challenge 

Collaboration 

Interdependence 

170 

Strategic  Advisory  Team 

Roads  were  often  poor 

Challenge 

Resources 

Roads 

171 

Strategic  Advisory  Team 

Difficult  to  get  travellers  back  to  the  compound  when  necessary  -  both  due  to  lack  of  vehicles  and 
poor  passability  of  roads 

Challenge 

Resources 

Roads 

172 

Strategic  Advisory  Team 

Difficult  to  get  travellers  back  to  the  compound  when  necessary  -  both  due  to  lack  of  vehicles  and 
poor  passability  of  roads 

Challenge 

Resources 

Transportation 

173 

Strategic  Advisory  Team 

Poor  communication  infrastructure  between  team  members  (local  cell  network) 

Challenge 

Communication 

Infrastructure 

174 

Strategic  Advisory  Team 

Poor  communication  infrastructure  between  team  members  (local  cell  network) 

Challenge 

Resources 

Communication  infrastructure 

175 

Strategic  Advisory  Team 

Limited  number  of  vehicles  so  travel  required  a  lot  of  coordination 

Challenge 

Resources 

Transportation 

176 

Strategic  Advisory  Team 

Differences  in  culture  (different  communication  clarity,  literacy  levels,  no  banking  system  to  support 
transactions) 

Challenge 

Culture 

Clarity 

177 

Strategic  Advisory  Team 

Differences  in  culture  (different  communication  clarity,  literacy  levels,  no  banking  system  to  support 
transactions) 

Challenge 

Culture 

Literacy 

178 

Strategic  Advisory  Team 

Differences  in  culture  (different  communication  clarity,  literacy  levels,  no  banking  system  to  support 
transactions) 

Challenge 

Culture 

Banking 

179  Strategic  Advisory  Team  Vague  mission  goal  ("do  it  their  way")  Challenge  Direction  Vague 


Had  to  function  in  a  situation  where  there  were  a  lot  of  different  organizations  who  had  different 
goals 


180  Strategic  Advisory  Team 


Challenge 


Collaboration 


Goal  conflict 


Initial  Order 

Scenario 

Challenge/training  recommendation 

Challenge  or  Training 

General  theme 

Subtheme 

181 

Strategic  Advisory  Team 

DM  organized  a  daily  meeting  to  support  team  coordination 

Training 

Collaboration 

Coordinating  work 

182 

Strategic  Advisory  Team 

DM  build  relationships  with  chaplains  at  nearby  US  base 

Training 

Collaboration 

Networking 

183 

Strategic  Advisory  Team 

Some  team  members  did  not  appropriately  communication  information  to  the  DM  in  a  timely  way 

Challenge 

Communication 

Timeliness 

184 

Strategic  Advisory  Team 

Large  amount  of  flexibility  required  for  mission  -  mission  command  rather  than  strict  hierarchy 

Training 

Direction 

Flexibility 

185 

Strategic  Advisory  Team 

One  goal  of  the  DM  was  to  build  relationships  -  facilitated  interactions 

Training 

Collaboration 

Networking 

186 

Strategic  Advisory  Team 

Had  to  balance  being  openly  military  with  trying  to  pass  as  civilian 

Challenge 

Goal  conflict 

Achieve  multiple  conflicting  objectives 

187 

Strategic  Advisory  Team 

Chain  of  command  was  fuzzy  -  what  aspects  of  DM's  mission  controlled  by  CEFCOM  vs.  TFA 

Challenge 

Direction 

Unclear  chain  of  command 

188 

Strategic  Advisory  Team 

Team  cohesion  issues  sometimes  arose  -  much  pressure  to  be  cohesive  as  the  team  lived  together 

Challenge 

Collaboration 

Team  cohesion 

189 

Strategic  Advisory  Team 

Security  issues  (e.g.,  riots) 

Challenge 

Location 

Security  requirements 

190 

Strategic  Advisory  Team 

Lack  of  sufficient  evacuation  plan 

Challenge 

Planning 

Lack  of  sufficient  planning 

191 

Strategic  Advisory  Team 

Challenging  to  mentor  a  person  in  an  embarrasing  situation  that  could  impact  their  career 

Challenge 

Collaboration 

Mentoring  difficulty 

192 

Strategic  Advisory  Team 

Need  to  establish  relationships  -  personality  is  key 

Training 

Personality 

Networking 

193 

Strategic  Advisory  Team 

Need  to  establish  relationships  -  personality  is  key 

Training 

Collaboration 

Networking 

194 

Strategic  Advisory  Team 

Need  to  get  different  HQs  talking  when  issues  arise 

Need  to  see  implications  of  situation  when  things  go  wrong  -  e.g.,  after  riot  saw  need  for  improved 
coordination  and  communication  between  their  team  and  nearby  embassies  etc.  in  case  evacuation 

Training 

Collaboration 

195 

Strategic  Advisory  Team 

etc.  required 

Training 

Planning 

Adapt  as  required 

Carefully  consider  whether  to  wear  civilian  or  military  clothing  on  this  type  of  mission  (strategic 
advisor)  -  would  have  avoided  unwanted  attention  at  strategic  level  if  didn't  wear  military  clothing 

196 

Strategic  Advisory  Team 

from  the  beginning 

Carefully  consider  whether  to  wear  civilian  or  military  clothing  on  this  type  of  mission  (strategic 
advisor)  -  would  have  avoided  unwanted  attention  at  strategic  level  if  didn't  wear  military  clothing 

Training 

Strategic  issues 

197 

Strategic  Advisory  Team 

from  the  beginning 

Training 

Goal  conflict 

Achieve  multiple  conflicting  objectives 

When  communicating  about  unpleasant  events  with  subordinates  make  sure  you  get  all  pertinent 

198 

Strategic  Advisory  Team 

information 

Training 

Communication 

Get  all  required  information 

Make  sure  you  have  information  about  road  passability  if  relevant  and  also  have  backup  plans  to  get 
people  home  (flight  cancelled),  give  travellers  resources  in  case  they  have  to  stay  away  longer  than 

199 

Strategic  Advisory  Team 

planned 

Make  sure  you  have  information  about  road  passability  if  relevant  and  also  have  backup  plans  to  get 
people  home  (flight  cancelled),  give  travellers  resources  in  case  they  have  to  stay  away  longer  than 

Training 

Planning 

Create  backup  plans 

200 

Strategic  Advisory  Team 

planned 

Training 

Planning 

Get  required  information 

Make  sure  as  much  as  possible  to  work  on  getting  everyone  to  work  as  a  team  -  even  one  person  who 

201 

Strategic  Advisory  Team 

is  not  a  team  player  affects  morale  and  decision  making 

Training 

Collaboration 

Team  cohesion 

202 

Strategic  Advisory  Team 

Try  to  make  decisions  as  a  team  as  much  as  possible 

Use  meetings  to  understand  everyone's  views  and  needs  as  well  as  practical  details  for  coordination 

Training 

Collaboration 

Teamwork 

203 

Strategic  Advisory  Team 

etc. 

Training 

Collaboration 

Teamwork 

Coordination  meetings  were  led  by  COS  rather  than  the  CO  -  this  is  not  common  -  allowed  for  more 

204 

Strategic  Advisory  Team 

openness  and  collegiality 

Training 

Collaboration 

Coordinating  work 

205 

Strategic  Advisory  Team 

Pick  the  right  person  for  the  team  -  right  amount  of  initiative,  passion,  and  skills 

Needed  to  properly  balance  mission  command  approach  with  C2  approach  to  get  needed  benefits  of 

Training 

Collaboration 

Right  person  in  right  job 

206 

Strategic  Advisory  Team 

both  (e.g.,  C2  constant  re-evaluation  of  plans,  mission  command  flexibility) 

Training 

Goal  conflict 

Achieve  multiple  conflicting  objectives 

207 

CoE  Training  Development 

Have  responsibility  but  no  real  authority 

Challenge 

Authority 

Responsibility  without  authority 

208 

CoE  Training  Development 

Trainees  respond  unpredictably  to  events  (what  they  will  choose  to  do  and  how  well  they  respond) 

Challenge 

Evaluation 

Unpredictability 

209 

CoE  Training  Development 

Difficult  to  evaluate  likely  results  of  trainee  actions 

Challenge 

Evaluation 

Objective  assessment  difficult 

210 

CoE  Training  Development 

Need  to  create  exercises  that  are  both  realistic  and  controlled  enough  -  difficult  to  balance 

Challenge 

Goal  conflict 

Achieve  multiple  conflicting  objectives 

Sometimes  events  that  could  (and  do)  actually  happen  in  operations  are  not  seen  as  realistic  by 

211 

CoE  Training  Development 

trainees  before  they  go  on  the  operation  -  perceived  vs.  actual  realism  have  to  be  balanced 

Challenge 

Goal  conflict 

Achieve  multiple  conflicting  objectives 

212 

CoE  Training  Development 

Some  aspects  of  training  are  difficult  to  mentor  (e.g.,  interview  training) 

Challenge 

Collaboration 

Mentoring  difficulty 

213 

CoE  Training  Development 

Required  resources  are  often  not  easily  available  and  have  to  be  built  from  scratch 

Generally  a  great  many  demands  on  trainee's  time:  this  means  sometimes  they  miss  scheduled 

Challenge 

Resources 

Availability  in  general 

214 

CoE  Training  Development 

training 

Challenge 

Resources 

Time 

215 

CoE  Training  Development 

When  training  program  began,  the  DM  was  working  "in  a  void"  -  didn't  really  know  what  was  needed 

Challenge 

Direction 

Vague 

Initial  Order  Scenario 

Challenge/training  recommendation 

The  DM  had  never  experienced  the  type  of  situation/meeting  he  was  trying  to  recreate  which  made 

Challenge  or  Training 

General  theme 

Subtheme 

216  CoE  Training  Development 

his  job  more  difficult 

SA  about  operational  environments  and  priorities  must  be  constantly  updated  and  changes  made  to 

Challenge 

Planning 

Lack  of  required  information 

217  CoE  Training  Development 

follow  them  in  training 

Challenge 

Information 

Updates 

218  CoE  Training  Development 

Took  time  to  build  the  necessary  trust  between  agencies 

Challenge 

Collaboration 

Trust  building 

219  CoE  Training  Development 

Networking  and  relationship  building  were  actively  encouraged  and  built  into  training  over  time 
Effective  collaboration  required  training  to  evolve  a  common  understanding  between  organizations 

Training 

Collaboration 

Networking 

220  CoE  Training  Development 

(e.g.,  of  OPP  process) 

Training 

Collaboration 

Common  understanding 

221  CoE  Training  Development 

Money  available  for  training  decreased  over  time,  requiring  restructuring 

Networks  of  people  were  built  as  training  developed  which  helped  in  the  acquisition  of  resources  for 

Challenge 

Resources 

Budget 

222  CoE  Training  Development 

training  and  for  getting  other  organizations  involved. 

Training 

Collaboration 

Networking 

223  CoE  Training  Development 

Personal  contacts  often  proved  more  useful  than  "official"  channels 

Training 

Collaboration 

Networking 

224  CoE  Training  Development 

Changes  are  difficult  to  make  within  a  training  cycle 

Importance  of  networking  and  relationship  building  -  night-out  dinner  most  important  part  of 

Challenge 

Planning 

Lead  time  required  is  too  long 

225  CoE  Training  Development 

workshop  as  you  learn  more  about  the  people  you  are  dealing  with  and  create  networks 

Training 

Collaboration 

Networking 

226  CoE  Training  Development 

Building  networks  and  developing  trust  is  the  most  important  part  of  his  job 

Training 

Collaboration 

Networking 

227  CoE  Training  Development 

Social  side  is  most  important  aspect  of  the  job  -  create  shared  values,  have  interpersonal  skills,  etc. 

Training 

Collaboration 

Networking 

228  CoE  Training  Development 

Need  to  have  good  IQ  and  EQ 

Training 

Collaboration 

Emotional  intelligence 

229  CoE  Training  Development 

Have  to  be  able  to  coerce  or  convince  people  to  give  you  what  you  need 

Training 

Collaboration 

Negotiation 

230  CoE  Training  Development 

Have  to  be  able  to  build  trust 

Training 

Collaboration 

Trust  building 

231  CoE  Training  Development 

How  you  connect  with  people  is  fundamental 

Training 

Collaboration 

Networking 

232  CoE  Training  Development 

Ability  to  negotiate  really  well  is  important  in  theatre 

Need  to  have  a  joint  lessons  learned  cell  -  shouldn't  wait  until  the  mission  is  done,  get  information  as 

Training 

Collaboration 

Negotiation 

233  CoE  Training  Development 

its  happening 

Training 

Information 

Lessons  learned 

234  CoE  Supervisor 

235  CoE  Supervisor 

Needs  to  be  overlap  and  continuity  in  training 

Government  announcements  can  force  unexpected  readjustments  in  training 

Changes  may  be  announced  but  details  not  known,  requiring  assumptions  must  be  made  so  that 

Challenge 

Challenge 

Planning 

Strategic  issues 

Interdependence 

236  CoE  Supervisor 

enough  planning  lead  time  is  available 

Challenge 

Planning 

Lack  of  required  information 

237  CoE  Supervisor 

Training  cannot  rely  on  templates  because  the  rate  of  change  is  too  high 

Challenge 

Planning 

High  rate  of  change 

238  CoE  Supervisor 

Outside  events  (e.g.,  changes  in  policy,  different  operational  events)  force  a  lot  of  change 

Challenge 

Planning 

Changes  dictated  by  outside  forces 

239  CoE  Supervisor 

Often  overlapping  requests  for  training  time  and  resources  which  require  on-line  resource  shifting 

Challenge 

Planning 

Resource  shifting 

240  CoE  Supervisor 

Long  lead  times  are  often  required  for  training  objectives  to  be  met 

Training  of  CF  personnel  have  to  include  training  them  to  interact  with  other  organizations  -  requires  a 

Challenge 

Planning 

Lead  time  required  is  too  long 

241  CoE  Supervisor 

lot  of  work  to  get  these  groups  adequately  involved 

Different  stakeholders  typically  have  different  jargon  and  different  ways  of  doing  things  which  make 

Challenge 

Collaboration 

Get  proper  people  involved 

242  CoE  Supervisor 

communication  and  collaboration  difficult 

Challenge 

Communication 

Jargon 

243  CoE  Supervisor 

Timelines  of  stakeholders  are  different 

Money  has  become  more  of  a  constraint  which  influences  other  resource  availability  (e.g.,  number  of 

Challenge 

Collaboration 

Timelines 

244  CoE  Supervisor 

people  who  can  be  involved  in  the  writing  boards) 

Challenge 

Resources 

Budget 

245  CoE  Supervisor 

Often  training  has  to  support  other  issues  like  basic  teamwork  training,  sort  out  SOPs,  etc. 

Training  scenarios  have  to  be  firmly  grounded  in  the  current  operational  state  because  trainees  are 
often  aware  of  the  situation  and  will  be  influenced  by  the  perceived  relevance  of  the  training  (as  well 

Challenge 

Planning 

Interdependence 

246  CoE  Supervisor 

as  the  relevance  actually  being  affected) 

Challenge 

Planning 

Updated  information  required 

247  CoE  Supervisor 

Use  social  networks  to  get  needed  personnel  resources 

Training 

Collaboration 

Networking 

248  CoE  Supervisor 

The  job  requires  people  with  personalities  which  can  accept  not  having  real  authority 

The  personnel  recruited  to  run  training  must  change  over  time,  due  to  current  experience  levels, 

Training 

Personality 

Authority 

249  CoE  Supervisor 

scheduling  conflicts,  etc. 

CF  staff  in  the  training  programs  change  cyclically  and  frequently  over  time  so  certain  things  have  to 
be  done  repeatedly  (e.g.,  building  relationships,  informing  people  about  what  works  and  what 

Challenge 

Collaboration 

Turnover 

250  CoE  Supervisor 

doesn't) 

Challenge 

Collaboration 

Turnover 

251  CoE  Supervisor 

Requests  to  inject  other  groups  into  training  can  cause  scheduling  and  other  resource  conflicts 

Challenge 

Resources 

Scheduling 

252  CoE  Supervisor 

Use  personal  networks  to  get  needed  people  involved 

Training 

Collaboration 

Networking 

253  CoE  Supervisor 

Ensure  that  people  creating  training  exercises  have  proper  and  recent  experience 

Training 

Experience 

Right  person  in  right  job 

Order 

Scenario 

Challenge/training  recommendation 

Challenge  or  Training 

General  theme 

Subtheme 

254 

CoE  Supervisor 

Identify  personnel/SME  needs  as  soon  as  possible  and  inform  the  relevant  groups  as  early  as  possible 
to  facilitate  their  involvement 

Training 

Collaboration 

Inform  collaborators  early  about  desired  involvement 

255 

CoE  Supervisor 

Identify  personnel/SME  needs  as  soon  as  possible  and  inform  the  relevant  groups  as  early  as  possible 
to  facilitate  their  involvement 

Training 

Planning 

Identify  needs  early 

256  CoE  Supervisor 

Need  to  know  how  Ottawa  works  -  different  from  the  tactical  level.  Dealing  with  public  servants,  the 
procurement  world,  etc. 

Training 

Experience 

Need  relevant  experience 

257  CoE  Supervisor 

If  required,  write  your  own  terms  of  reference  for  yourself  and  your  staff 

Training 

Direction 

Need  to  create  clarity 

258  CoE  Supervisor 

Must  be  flexible  to  adapt  to  new  situations  and  deal  with  ambiguity 

Training 

Planning 

Flexibility 

259  CoE  Supervisor 

Can't  have  type-A  personalities  in  full-time  CoE  positions  because  they  may  butt  heads  with  the 
military 

Training 

Personality 

Type  A 

260  CoE  Supervisor 

Need  to  coordinate  as  much  as  possible  with  higher  command  (e.g.,  CANOSCOM)  to  get  information 
needed  to  form  training  objectives 

Training 

Direction 

Get  needed  information 

261  PSYOPS  training 

It  was  the  first  time  such  a  thorough  training  program  was  being  developed  -  previous  courses  were 
not  available  to  use  to  guide  planning 

Challenge 

Planning 

Lack  of  previous  similar  events 

262 

PSYOPS  training 

Short  timeline  to  plan 

Challenge 

Planning 

Short  timeline 

263 

PSYOPS  training 

Short  timeline  to  plan 

Challenge 

Resources 

Short  timeline 

264 

PSYOPS  training 

Because  of  short  planning  timeline  changes  in  one  resource  meant  that  other  changes  had  to  be  made 
to  accommodate  it  (e.g.,  instructor  availability  changes  meant  that  the  order  of  course  material  might 
have  to  change 

Challenge 

Resources 

Balancing 

265 

PSYOPS  training 

Some  elements  of  planning  occurred  before  the  DM  took  over  which  meant  they  were  not  in  his 
control  or  of  his  choosing 

Challenge 

Planning 

Lack  of  control 

266 

PSYOPS  training 

Pressure  to  have  the  trainees  ready  to  go  right  after  training  (ready  to  prove  their  worth) 

Challenge 

Planning 

High  stakes 

267 

PSYOPS  training 

Trainees  were  a  diverse  group  of  people  with  very  different  backgrounds  and  levels  of  military 
experience  (although  none  were  new  soldiers)  -  note  that  this  was  both  a  challenge  and  an  asset 

Challenge 

Collaboration 

Diverse  experience 

268 

PSYOPS  training 

Demands  by  the  training  course  location  to  not  have  the  trainees  in  uniform 

Challenge 

Location 

269 

PSYOPS  training 

Have  and  use  a  network  of  resources  through  personal  contacts  etc.  to  provide  training  opportunities 

Training 

Collaboration 

Networking 

270 

PSYOPS  training 

Take  advantage  of  local  resources  as  much  as  possible  (e.g.,  training  located  where  there  is  a  large 
Afghan  community  so  could  get  them  involved  as  actors  etc.) 

Training 

Planning 

Use  available  resources 

271 

PSYOPS  training 

Instructor  schedules  produced  constraints  on  information  delivery  -  information  was  not  delivered  in 
optimal  order 

Challenge 

Resources 

Scheduling 

272 

PSYOPS  training 

Take  care  of  group  cohesion 

Training 

Collaboration 

Team  cohesion 

273 

PSYOPS  training 

Planning  had  to  remain  an  ongoing  process  during  the  training  itself 

Challenge 

Planning 

Ongoing 

274 

PSYOPS  training 

Be  aware  that  some  of  the  benefits  of  training  may  not  be  appreciated  at  the  time 

Training 

Evaluation 

Time  delay 

275 

PSYOPS  training 

Need  to  be  prepared  (and  prepare  trainees)  to  justify  their  role  -  often  PSYOPS  is  not  understood  (e.g. 
get  an  order  to  "go  PSYOPS  those  guys  and  be  done  in  an  hour"  or  appreciated  -  need  to  be  able  to 
relate  to  a  strategic/political  end  goal 

Training 

Role  justification 

276 

PSYOPS  training 

Need  to  be  able  to  do  job  and  give  limelight  to  the  people  who  feel  they  deserve  it 

Training 

Collaboration 

Credit 

277 

PSYOPS  training 

Can  learn  more  from  interacting  with  the  actual  civilian  community  than  you  can  from  training 

Training 

Information 

Real  world 

278 

PSYOPS  training 

Had  go/no  go  criteria  for  each  exercise 

Training 

Planning 

Use  go/no  go  criteria 

279 

PSYOPS  training 

Some  instructors  gave  too  much  information  too  quickly  and  at  too  high  a  level  -  took  time  to  absorb 
after  training 

Challenge 

Information 

Overload 

280 

PSYOPS  training 

Attempted  to  integrate  other  needed  skills  into  PSYOPS  training  to  increase  their  usefulness  -  close 
quarter  combat  training,  combat  casualty  care 

Training 

Planning 

Interdependence 

281 

Logistics  -  NSE 

Logistics  staff  and  other  resources  were  extremely  limited 

Challenge 

Resources 

Staff 

282 

Logistics  -  NSE 

CONOPS  required  dispersed  logistics  whereas  for  logistics  it  is  always  easier  to  be  centralized 

Challenge 

Resources 

Location 

283 

Logistics  -  NSE 

Difficult  to  move  resources  (locations  far  apart;  difficult  terrain;  requirement  to  travel  through  areas 
inhabited  by  the  enemy) 

Challenge 

Resources 

Location 

284 

Logistics  -  NSE 

Logistics  extremely  brittle  and  vulnerable  to  unexpected  events  (due  to  lack  of  resources  etc.) 

Challenge 

Planning 

Vulnerability  to  the  unexpected 

285 

Logistics  -  NSE 

Individual  differences  in  BG  members  meant  that  it  was  difficult  to  track  resource  usage  (e.g.,  rate  of 
artillery  usage) 

Challenge 

Collaboration 

Individual  differences 

286 

Logistics  -  NSE 

Individual  differences  in  BG  members  meant  that  it  was  difficult  to  track  resource  usage  (e.g.,  rate  of 
artillery  usage) 

Challenge 

Information 

Updates 

287  Logistics  -  NSE 

Enemy  actions  were  unpredictable 

Challenge 

Planning 

Unpredictability 

288  Logistics  -  NSE 

Logistics  considered  secondary  to  combat  forces 

Challenge 

Role  justification 

289  Logistics  -  NSE 

Time  lags  between  resource  requests  and  replenishment  from  Canada 

Challenge 

Resources 

Replenishment 

Initial  Order 

Scenario 

Challenge/training  recommendation 

Challenge  or  Training 

General  theme 

Subtheme 

290 

Logistics  -  NSE 

Very  limited  logistics  staff 

Challenge 

Resources 

Staff 

291 

Logistics  -  NSE 

Need  to  consider  resources  of  allies 

Challenge 

Collaboration 

Resources 

292 

Logistics  -  NSE 

Lack  of  information  to  support  planning  (mission  type  relatively  different  from  recent  missions) 

Challenge 

Planning 

Lack  of  previous  similar  events 

293 

Logistics  -  NSE 

Political/strategic  concerns  limited  resource  options 

Challenge 

Strategic  issues 

294 

Logistics  -  NSE 

Environmental  effects  (e.g.,  maintenance  more  frequent) 

Location  of  resources  (e.g.,  FOBs  and  contents)  change  relevance  based  on  actions  of  the  enemy  and 

Challenge 

Location 

Harsh  conditions 

295 

Logistics  -  NSE 

orders  from  higher  command  (e.g.,  where  to  deploy) 

Challenge 

Resources 

Location 

296 

Logistics  -  NSE 

Logistics  could  not  influence  locations  of  FOBs  but  had  to  keep  them  supplied 

Challenge 

Planning 

Lack  of  control 

297 

Logistics  -  NSE 

Psychological  well-being  of  logistics  staff  endangered  due  to  lack  of  sleep  and  rest 

True  sustainment  not  practiced  by  BG  -  led  to  lack  of  information  for  planners  and  lack  of  prep  for 

Challenge 

Resources 

Sleep  and  rest 

298 

Logistics  -  NSE 

soldiers 

Location  of  conflict  (Afghanistan)  had  large  impact  on  flexibility  to  replenish  resources  (e.g.,  land- 

Challenge 

Planning 

Lack  of  required  information 

299 

Logistics  -  NSE 

locked  country) 

Challenge 

Location 

Lack  of  flexibility 

Radical  change  to  CONOPS  that  the  DM  was  unaware  of  prior  to  deployment  had  profound  negative 

300 

Logistics  -  NSE 

effects  on  logistics  (e.g.,  went  from  centralized  to  decentralized  logistics) 

Challenge 

Planning 

Lack  of  required  information 

The  use  of  resources  varied  greatly  over  time  and  made  it  very  difficult  to  keep  track  of  when 

301 

Logistics  -  NSE 

replenishment  needed 

The  use  of  resources  varied  greatly  over  time  and  made  it  very  difficult  to  keep  track  of  when 

Challenge 

Information 

Updates 

302 

Logistics  -  NSE 

replenishment  needed 

Challenge 

Resources 

Information 

303  Logistics  -  NSE 

Have  to  be  able  to  handle  whatever  level  of  independence  given  by  higher  command 

Training 

Personality 

Independence 

304  Logistics  -  NSE 

Need  to  understand  and  know  how  to  deal  with  the  personalities  of  those  you  have  to  work  with 

Training 

Collaboration 

Personality 
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Acronyms 


Acronym 

Full  Term 

AAR 

After  Action  Review 

ANA 

Afghan  National  Army 

ANP 

Afghan  National  Police 

ATL 

Adaptive  Thinking  and  Leadership 

BG 

Battle  Group 

C2 

Command  and  Control 

CANOSCOM 

Canadian  Operational  Support  Command 

CAS 

Complex  Adaptive  Systems 

CDA 

Canadian  Defence  Academy 

CDM 

Critical  Decision  Method 

CEFCOM 

Canadian  Expeditionary  Force  Command 

CF 

Canadian  Forces 

CFC 

Canadian  Forces  College 

Cl  DA 

Canadian  International  Development  Agency 

CIMIC 

Civil  Military  Cooperation 

CLFCSC 

Canadian  Land  Force  Command  and  Staff  College 

CMP 

Chief  of  Military  Personnel 

CoE 

Centre  of  Excellence 

COIN 

Counter  Insurgency 

CONOPS 

Concept  of  Operations 

COP 

Common  Operating  Picture 

DFAIT 

Department  of  Foreign  Affairs  and  International  Trade 

DL 

Distance  Learning 

DM 

Decision  Maker 

DND 

Department  of  National  Defence 

DRDC 

Defence  Research  and  Development  Canada 

EC 

Emotional  Quotient 

FOB 

Forward  Operating  Base 

HF 

Human  Factors 

HSI® 

HumanSysfems  Incorporated 
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Acronym 

Full  Term 

HQ 

Headquarters 

IEC 

Independent  Electoral  Commission 

IPP 

Intuitive  Planning  Process 

ISAF 

International  Security  Assistance  Force 

LAV(s) 

Light  Armoured  Vehicle(s) 

LCC 

Local  Command  Centre 

NDS 

National  Directorate  of  Security 

NGO 

Non-Governmental  Organization 

NSE 

National  Support  Element 

OPP 

Operational  Planning  Process;  Ontario  Provincial  Police 

PD 

Professional  Development 

PMB 

Program  Management  Board 

PME 

Professional  Military  Education 

PoC(s) 

Point(s)  of  Contact 

PSYOPS 

Psychological  Operations 

R&D 

Research  and  Development 

RCMP 

Royal  Canadian  Mounted  Police 

SA 

Scientific  Authority 

SOP 

Standard  Operating  Procedure 

SME(s) 

Subject  Matter  Expert(s) 

TFA 

Task  Force  Afghanistan 

TLCTS 

Tactical  Iraqi  Language  and  Culture  Training  System 

UN 

United  Nations 
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Glossary 


Term 

Definition 

Complexity 

According  to  Dorner  (1996),  the  more  variables  in  a 
system  and  the  greater  their  independence,  the  more 
complex  that  system  is. 

Connectivity 

A  factor  that  influences  the  complexity  of  a  decision 
making  situation.  The  extent  to  which  things  in  the 
environment  influence  one  another  in  complicated  and 
unpredictable  ways. 

Critical  Decision  Method 

The  Critical  Decision  Method  (or  CDM)  is  a  method  of 
knowledge  elicitation  that  focuses  the  interviewee  on  an 
incident  in  their  experience  which  contained  a  critical 
decision  related  to  the  topic  under  discussion.  The 
interviewee  is  asked  to  elaborate  on  their  experience  to 
elicit  information  of  interest  to  the  interviewer. 

Dynamics 

A  factor  that  influences  the  complexity  of  a  decision 
making  situation.  The  extent  to  which  the  system  has 
aspects  that  unfold  over  time.  For  example,  the 
environment  changes  over  time  even  when  you  do 
nothing;  the  rate  at  which  things  change  may  be  variable; 
there  may  be  delays  between  actions  and  effects. 

Independent  agents 

A  factor  that  influences  the  complexity  of  a  decision 
making  situation.  The  extent  to  which  there  are 
independent  entities  in  the  environment  who  influence  it 
(they  may  have  different  goals  than  the  decision  maker). 

Microworlds 

Microworlds  are  computer  simulations  of  complex 
environments.  They  generally  allow  interaction  and  are 
used  to  examine  the  effectiveness  with  which  people  can 
interact  with  complex  and  dynamic  domains. 

Multiple  conflicting  goals 

A  factor  that  influences  the  complexity  of  a  decision 
making  situation.  The  extent  to  which  the  DM  has  to 
achieve  multiple  objectives  which  may  not  be  achievable 
at  the  same  time. 

Underspecified  goals 

A  factor  that  influences  the  complexity  of  a  decision 
making  situation.  The  extent  to  which  goals  may  be  difficult 
to  achieve  because  they  are  too  vague. 
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